17th July → Nirmal

Updates to Design

  1. Let’s have 3 models

    1. Blockchain

      1. name (ex: bitcoin, ethereum)

    2. Provider

      1. name (ex: blockdaemon, tatum, getblock)

      2. domain (ex: getblock.io, tatum.io, blockdaemon.com)

      3. is_enabled (true / false)

    3. ProviderIntegration

      1. This is the existing model that you’ve created, replace text fields with Foreign keys to above models

      2. Add an integer field called priority (1 is highest, 2 is lower and so on)

      3. Use django’s unique_together to ensure records are unique on (blockchain, priority)
        https://docs.djangoproject.com/en/4.0/ref/models/options/#unique-together

      4. The URL provided in this object must contain provider.domain (data integrity check)

  2. Admin Interfaces

    1. Blockchain ModelAdmin

      1. Show blockchain name

      2. InlineTabularAdmin to show related ProviderIntegration objects. I should be able to add and edit providers from this admin interface, other admin’s are not required

      3. I can add a new blockchain without adding providerintegration objects

    2. Provider ModelAdmin

      1. Add a provider → This will rarely be used except in the first few days

      2. I can disable a provider from here. When I do this, all providerIntegrations related to this provider will not be used to calculate priority

Functional Changes

  1. In the event, two ProviderIntegrations have the same block_number (and it is the latest block number), you can resolve this clash using priority

  2. Incoming requests must contain an apikey GET parameter which can be used to authenticate requests. Something on these lines

    from rest_framework import permissions

class IsInternalApiRequest(permissions.BasePermission):
    def has_permission(self, request, view):
        if request.GET("X-API-KEY") == config("ETL-API-KEY", cast=str, default="empty"):
            return True
        return False
    CODE

Use this permission class to authenticate your request

Store ETL-API-KEY on doppler, it can be a constant

  1. Create a healthcheck endpoint called /healthcheck/ping which will return 200 if the backend service is live

  2. Create a liveness endpoint called live /healtcheck/liveness which will always return a 200 OK response but in message body also returns (current_time - time_cache_was_last_updated) in seconds. We will use this endpoint to identify if i) sauron is not providing latest blocks ii) if you’re task which updates latest blocks has gone down

  3. Could you write docs for local setup + how to use doppler (refer to goldeneye documentation)
    https://github.com/merklescience/goldeneye

  4. Add code linting checks - flake8, black (we have an implementation, check with us)

  5. Deploy to google app engine

  6. Create postman collection

  7. Setup alerts on BetterUptime

  8. Setup monitoring via stitchdata and tableau

  9. Build in CI / CD pipeline

  10. Install new relic

  11. Think about error handling

  12. Setup dev environment

  13. Add unit tests