Get URL endpoints from another field than the id

Please read this tip until the end because I describe all errors I committed before to find a good solution. Or just go to see Final codes below.

OK, I wanted update data from a tool (tool A), after users updates from another tool (tool B). The perfect field of action for Django REST framework I guess. So I built my own API with Django 3.

But I faced a problem: the relationship between the data had to be on the email field. And by default Django REST framework uses ids to create URL endpoints of each record. Indeed it seems to be this kind of URLs which allow updates by record (PUT command with httpie for example).

These are the ones I need to include in the code that will run in the forms of tool B!

Example of httpie command to update fields through a REST API

> http -a UserLogin:PassWord PUT http://127.0.0.1:8000/users/4/ firstname="Gretha" lastname="Thunberg"

The URL contained above is the endpoint of a record, using its id (4) from tool A. But tool B does not know this id, and does not have to know it. He just want to update firstname and lastname according an email relationship. Kwarghhh!

First I tried to generate URL endpoints with emails, hoping something like:

... http://127.0.0.1:8000/users/greta-thunberg@earth.com/ ...

I know, it's weird. No matter, I never succeeded.

So I had to find a way to get the good URL endpoints without knowing their ids, comparing emails.

Email field and URL endpoint for each record being provided in the json from my API, I use the code below:

1. Json authentication, then reading
2. Json querying to recover URL endpoint according email value
3. Build of the PUT command, then execution

import requests, json, subprocess
 
REQUEST_URL = 'http://127.0.0.1:8000/users/?format=json'
login = 'DjangoLogin'
password = 'DjangoPassWord'
response = requests.get(REQUEST_URL, auth=(login, password))
 
json_data = response.text.encode('utf-8', 'ignore')
readable_json = json.loads(json_data)
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
match_count = 0
 
for results in readable_json['results']:
    match_count += 1
    if results['email'] == email_reference and results['email'] is not None and match_count != 1:
        my_url = results['url']
 
        my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="' + new_firstname + '"' + ' lastname="' + new_lastname + '"'
 
        p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = p.communicate()

I check that email is not empty (is not None), and I add an iteration to retrieve only the first result of the data from tool A (match_count). It will also be better for your tests.

But with huge data ...

To work our json must display all your data. Indeed for now we search the email value into all the dataset. What if you store a lot of data?

Add a pagination, in setting.py, for example only 1000 records by page:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',
    'PAGE_SIZE': 1000
}

OK, if you refresh and test, pagination from our json appears in URLs with the final &offset=NUMBER, where NUMBER is the record from you want to start to displaying.

So now you have to iterate through your json, until 500 000 for example:

import requests, json, subprocess
 
login = 'DjangoLogin'
password = 'DjangoPassWord'
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
REQUEST_URL_PART = 'http://127.0.0.1:8000/users/?format=json&offset='
 
match_count_A = 0
while match_count_A < 500000:
    REQUEST_URL = REQUEST_URL_PART + str(match_count_A)
    response = requests.get(REQUEST_URL, auth=(login, password))
    json_data = response.text.encode('utf-8', 'ignore')
    readable_json = json.loads(json_data)
    match_count_A += 1000
 
    match_count_B = 0
    for results in readable_json['results']:
       match_count_B += 1
        if results['email'] == email_reference and results['email'] is not None and match_count_B != 1:
            my_url = results['url']
 
            my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="' + new_firstname + '"' + ' lastname="' + new_lastname + '"'
 
            p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
            out, err = p.communicate()

I know, it's still weird ...

Humm ... Even with this, I tried with a huge database ... Not really the good solution. Run through the json can be very long.

Final codes

No, I better to filter my json to find the well record before any while or update. It is possible creating another view in our API to access records filtering fields in URLs, email field for us.

A def get_queryset in yourapp/views.py :

from django.shortcuts import render
from .models import YourModel
 
from rest_framework import viewsets
from rest_framework import permissions
 
from YourApp.serializers import YourSerializer
 
class YourViewsetForEmail(viewsets.ModelViewSet):
    queryset = YourModel.objects.all()
    serializer_class = YourSerializer
    permission_classes = [permissions.IsAdminUser]
 
    def get_queryset(self):
        email = self.request.query_params.get('email', None)
        return YourModel.objects.filter(email=email)

In your project/urls.py:

from django.contrib import admin
from django.urls import include, path
from rest_framework import routers
from YourApp import views
 
router = routers.DefaultRouter()
router.register(r'users', views.YourFirstAPIViewset)
router.register(r'users_by_email', views.YourViewsetForEmail)
 
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]

I note this will create an aesthetic problem in the URLs mentioned in the json, but it is only aesthetic (true URLS still work), I will fix it later.

OK, now this URL displays the well record: http://127.0.0.1:8000/users_by_email/?email=Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.

In json: http://127.0.0.1:8000/users_by_email/?format=json&email=Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.

And the record endpoint (mandatory to update) still is: http://127.0.0.1:8000/users/ID/

So an example of a final code to update records through our API according their email will be:

import requests, json, subprocess
 
login = 'DjangoLogin'
password = 'DjangoPassWord'
 
email_reference = YOUR_EMAIL_FIELD_TOOL_B
new_firstname = YOUR_FIRSTNAME_FIELD_TOOL_B
new_lastname = YOUR_LASTNAME_FIELD_TOOL_B
 
REQUEST_URL_PART = 'http://127.0.0.1:8000/users_by_email/?format=json&email='
REQUEST_URL = REQUEST_URL_PART + email_reference
 
response = requests.get(REQUEST_URL, auth=(login, password))
json_data = response.text.encode('utf-8', 'ignore')
readable_json = json.loads(json_data)
 
for results in readable_json['results']:
    if results['email'] == email_reference and results['email'] is not None:
 
        my_id = results['id']
        my_url = 'http://127.0.0.1:8000/users/' + str(my_id) + '/'
 
        my_cmd = 'http -a ' + login + ':' + password + ' PUT ' + my_url + ' firstname="'+ new_firstname +'" lastname=" ' + new_lastname + '"'
 
        p = subprocess.Popen(my_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        out, err = p.communicate()