Austin Riba 5 Mar 2024 && [ code ] && 0 comments

When and why did MIT become the default license for open source projects?

I was watching some Fireship videos yesterday and one popped up called The Dark Side of Open Source // What really happened to Faker.js? I am familiar with the story of Aaron Swartz, but I had not heard about faker.js. The author of this extremely popular library got so fed up with companies taking and giving nothing in return that he blanked out the repo and broke it for everyone.

I couldn’t help but notice faker.js was under the MIT license.

If you don’t want for profit companies to use your code and give nothing in return, which is the modus operandi of pretty much every company, don’t use the MIT license.

Copyleft licenses like the GPL exist for a reason. If you don’t want companies to exploit your work for profit, use them!

⚙ ⚙ ⚙

Austin Riba 3 Mar 2024 && [ code, memes ] && 0 comments

I think I’m going to try Zig

⚙ ⚙ ⚙

Austin Riba 28 Feb 2024 && [ code ] && 0 comments

I was playing around with Web Components and thought it would be neat to be able to display code snippets, but instead of static text load the content from a remote source. For example, displaying the source code for a file hosted on Github inline in a blog post. Thus prism-remote was born.

For example: To highlight lines 1 through 20 of https://github.com/Fingel/prism-remote/blob/main/prism-remote.js using the <prism-remote> custom element:

<prism-remote
    src="https://github.com/Fingel/prism-remote/blob/main/prism-remote.js"
    start="1"
    end="20"
    lang="javascript"
>
</prism-remote>

Results in:

While I think the Webcomponent API leaves a little to be desired, there are clearly neat use cases like this where they can be very useful.

⚙ ⚙ ⚙

I’ve been using Kagi for a month and I haven’t missed Google for a minute. Crazy to me how it can absolutely demolish one of the world’s most powerful company’s flagship products.

Austin Riba 26 Jan 2024 && [ code, django, python ] && 6 comments

In this post, I’m going to explore using Django, HTMX, django-components and a slick project layout to build the best todo app you’ve ever seen that doesn’t require React.

HTMX seems to be having a moment and not just because the X account is a top-tier meme factory. It seems to fill a nice somewhere between full luddite server side rendering and the insanity of whatever is currently going on in the frontend land (it’s Next.js right? Or are we already on to the next… js? 😮‍💨)

Django seems like a perfectly fine framework to use with HTMX and there are plenty of tutorials out there already on this subject. Enter django-components, a neat little library I was recently turned on to that promises to deliver some of the nice parts of modern web dev: isolated, re-usable UI components, but for Django.

The source code for this demo is located in the Github repo.

One last thing to note: I’m using a simplified django project layout which is ideal for little demo purposes like this. No need to create an app when you know there will only be one. The project layout looks like this:

../dj_super_todo/
├── asgi.py
├── db.sqlite3
├── dj_super_todo
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── manage.py
├── settings.py
├── urls.py
└── wsgi.py

First create the simplest TODO model possible:

# models.py
from django.db import models


class Todo(models.Model):
    done = models.BooleanField(blank=True, default=False)
    content = models.CharField(max_length=500)

Create the migrations and seed the database with a few TODOs using the normal Django methods.

./manage.py makemigrations
./manage.py migrate
./manage.py shell
>>> Todo.objects.create(content="Get better at speaking to my dog.")
<Todo: Todo object (1)>
>>> Todo.objects.create(content="Get up, get up, and get down.")
<Todo: Todo object (2)>

Next install django-components and follow the installation instructions.

Now create the simplest todo component possible:

The template: components/todo/todo.html

<div class="todo" id="todo-{{ todo.id }}">
    {{ todo.content }}
    <button>Done</button>
</div>

The CSS: components/todo/todo.css

.todo {
    width: 300px;
    background: skyblue;
    border: 1px solid;
    margin: 2px 0px;
    padding: 2px;
}

And register the component in components/todo/todo.py

from django_components import component


@component.register("todo")
class Todo(component.Component):
    template_name = "todo/todo.html"

    def get_context_data(self, todo):
        return {"todo": todo}

    class Media:
        css = "todo/todo.css"

Now that the todo component is defined we can use it in a template that lists all todos. Add BASE_DIR / "templates" to settings.TEMPLATES.DIRS so we can add templates/index.html that lists our todos:

<!doctype html>
<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>Best TODO app</title>
        {% component_css_dependencies %}
        <script
            src="https://unpkg.com/htmx.org@1.9.10"
            integrity="sha384-D1Kt99CQMDuVetoL1lrYwg5t+9QdHe7NLX/SoJYkXDFfX37iInKRy5xLSi8nO7UC"
            crossorigin="anonymous"
        ></script>
    </head>
    <body hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'>
        <div id="todos">
            {% for todo in todos %} {% component "todo" todo=todo %} {% endfor
            %}
        </div>
        {% component_js_dependencies %}
    </body>
</html>

Notice the for loop which uses our todo component and passes in the single todo object.

In views.py, we define a very simple view that renders this template:

def index(request):
    todos = Todo.objects.all()
    return render(request, "index.html", {"todos": todos})

And add this view to urls.py:

urlpatterns = [
    path("", index, name="index"),
    path("admin/", admin.site.urls),
]

Now upon running and visiting our new django app this should appear:

Nice! Already we begin to see the power of django-components. But a todo app isn’t very useful if you can’t add new ones on which you can perpetually procrastinate.

Components defined with django-components have an interesting feature in that they inherit from Django’s View class. Which means they can respond to HTTP requests. Add a post method to the todo component which creates a new Todo object. It then returns itself as rendered HTML in the response which turns out to be perfect for consumption with HTMX as demonstrated further on.

Also a delete method is added.

from django.http import HttpResponse
from django_components import component

from dj_super_todo.models import Todo


@component.register("todo")
class TodoComponent(component.Component):
    template_name = "todo/todo.html"

    def get_context_data(self, todo):
        return {"todo": todo}

    def post(self, request, *args, **kwargs):
        todo = Todo.objects.create(content=request.POST["content"])
        return self.render_to_response({"todo": todo})

    def delete(self, request, pk, *args, **kwargs):
        Todo.objects.get(pk=pk).delete()
        return HttpResponse()

    class Media:
        css = "todo/todo.css"

Edit urls.py to make routes for these new views. As a Component is a subclass of a Django View it is possible to use their as_view() functions:

from django.contrib import admin
from django.urls import path

from components.todo.todo import TodoComponent
from dj_super_todo.views import index

urlpatterns = [
    path("components/todo", TodoComponent.as_view(), name="todos"),
    path("components/todo/<int:pk>/", TodoComponent.as_view(), name="todo"),
    path("", index, name="index"),
    path("admin/", admin.site.urls),
]

Finally modify both index.html and todo.html with some HTMX to create new Todo and delete them, respectively.

<!doctype html>
<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>Best TODO app</title>
        {% component_css_dependencies %}
        <script
            src="https://unpkg.com/htmx.org@1.9.10"
            integrity="sha384-D1Kt99CQMDuVetoL1lrYwg5t+9QdHe7NLX/SoJYkXDFfX37iInKRy5xLSi8nO7UC"
            crossorigin="anonymous"
        ></script>
    </head>
    <body hx-headers='{"X-CSRFToken": "{{ csrf_token }}"}'>
        <div id="todos">
            {% for todo in todos %} {% component "todo" todo=todo %} {% endfor
            %}
        </div>
        <form
            hx-post="{% url 'todos' %}"
            hx-swap="beforeend"
            hx-target="#todos"
        >
            {% csrf_token %}
            <input type="text" name="content" />
            <button type="submit">Add Todo</button>
        </form>
        {% component_js_dependencies %}
    </body>
</html>

Notice the form. The hx-post attribute points to our view that calls the post method on our Todo Component. hx-target tells HTMX to place the response content in the #todos div, and htx-swap=beforeend instructs HTMX to put the content after the last child element of the target. This adds another Todo to the list without a page refresh.

In todo.html we add some HTMX to the individual component to call the delete view:

<div class="todo" id="todo-{{ todo.id }}">
    {{ todo.content }}
    <button
        hx-delete="{% url 'todo' pk=todo.id %}"
        hx-target="#todo-{{todo.id}}"
        hx-swap="outerHTML"
    >
        Done
    </button>
</div>

Similar to the markup for post, hx-delete attribute points to the delete view, hx-target defines which element to replace, and hx-swap="outerHTML tells HTMX to replace the entire element with the response, which is nothing.

That’s it! We now have a TODO app that feels like something written with a traditional JS framework, but in fact we didn’t have to write any Javascript at all (though we could have if we wanted). Coupled with Django-components and their ability to act as views, we have a nice model for isolated and re-usable components that feels quite elegant.

To view the source code as a complete project, check out the Github repo.

⚙ ⚙ ⚙

Austin Riba 23 Sep 2023 && [ linux ] && 3 comments

What is the simplest TODO app imaginable? In my opinion, it’s just a text file in your home directory named todo.txt. One line per item, edited with Vim. No need for additional software, websites, or electron apps.

Using standard unix tools, this setup is easy to extend. In my case, I wanted a persistent, nagging reminder of my constant procrastination. I also use waybar. Naturally then the end goal is a custom module.

Obviously, it displays the number of TODOs you have remaining. Additionally, hovering over the module will print display them in a tooltip. When you click a new Neovim instance will spawn opening the file.

Simple and effective. Here’s the code. I placed this is ~/.config/waybar/scripts/todo.sh:

#!/bin/bash
COUNT=$(wc -l < ~/todo.txt)
TODOS=$(cat ~/todo.txt | head -c -1 - | sed -z 's/\n/\\n/g')
printf '{"text": "%s", "tooltip": "%s"}\n' "$COUNT" "$TODOS"

And then add a custom module to ~/.config/waybar/config:

 "custom/todo": {
        "exec": "~/.config/waybar/scripts/todo.sh",
        "return-type": "json",
        "format": "{} todos",
        "on-click": "wezterm -e nvim ~/todo.txt",
        "interval": 5,
    }

Replace wezterm with your preferred terminal emulator and you should be good to go!

⚙ ⚙ ⚙

Austin Riba 25 Sep 2021 && [ code, python ] && 0 comments

As a developer, you want the APIs you write to be as fast as possible. So what if I told you that with this one simple trick, you might be able to increase the speed of your API by 2x, 3x, or maybe even 4x? You’d probably tell me to get lost with the clickbait, but hear me out. In this article you will learn how to utilize Python asyncio, the httpx library, and the Flask micro framework to optimize certain parts of your API.

In this tutorial you will:

1. Write a small HTTP API using everyone’s favorite Python framework: Flask.
2. Use httpx, an awesome modern Python HTTP client that supports async.
3. Familiarize yourself with a small subset of Python’s asyncio Library.

One app, two endpoints.

To begin this feat of strength, you will write a simple Flask app with two endpoints. One will be asynchronous and the other will not.

For maximum compatibility, please make sure you are using Python 3.8 or newer.

Begin by creating a directory to hold your code and create a virtual environment in it:

mkdir asyncapi
cd asyncapi
python3 -m venv env/

Activate the virtual environment and install Flask with async support.

source venv/bin/activate
pip install Flask[async]

Next, place the following code in a file named app.py.

from flask import Flask

app = Flask(__name__)

@app.route('/get_data')
def get_data():
    return 'ok'

@app.route('/async_get_data')
async def async_get_data():
    return 'ok'

You now have a fully working Flask API with two endpoints. If you are new to Flask and are curious what is going on here, check out the Flask quickstart documentation.

Notice that the second endpoint, /async_get_data uses the async def syntax for defining it’s method. Still, this endpoint does exactly the same thing as the normal /get_data endpoint, except that we can run asynchronous code in it. As it is written now, however, it is not any faster. We can prove this by running our API and making a few calls with cURL.

Start the Flask development server:

flask run

Now time some cURL requests to show that both endpoints return in about the same time:

time curl "http://localhost:5000/get_data"
ok
________________________________________________________
Executed in    7.28 millis    fish           external
   usr time    5.86 millis  248.00 micros    5.61 millis
   sys time    0.03 millis   32.00 micros    0.00 millis

Notice the line that says “Executed in 7.28 millis”. That’s pretty quick. Try again using the other endpoint:

time curl "http://localhost:5000/async_get_data"
ok
________________________________________________________
Executed in   21.77 millis    fish           external
   usr time    2.48 millis    0.00 micros    2.48 millis
   sys time    2.77 millis  293.00 micros    2.48 millis

Not only was the async version not faster, it was about 3x slower. The difference between 7 miliseconds and 21 miliseconds is not noticeable to our human eyes, But this is a good demonstration that there can be overhead to using asyncio, so it is not faster in all situations.

Two endpoints, one fast, one slow.

In order to see the async_get_data endpoint become faster than it’s sync counterpart, you’ll have to make the endpoints actually do some work. One common case for APIs is that they need to make calls to other APIs, for example, to fetch the weather for a specific location from a third party service.

You can add HTTP requests to your API using a combination of httpx and Flash, a service that intentionally returns slow HTTP responses. Why slow? Because you want to be able to simulate large and or slow external APIs, as well as exaggerate the effect of using asyncio.

First, install the httpx library:

pip install httpx

Modify app.py to look like this:

from flask import Flask
import asyncio
import httpx

app = Flask(__name__)

@app.route('/get_data')
def get_data():
    r1 = httpx.get('https://flash.siwalik.in/delay/1000/')
    r2 = httpx.get('https://flash.siwalik.in/delay/1000/')

    return {
        'r1': r1.status_code,
        'r2': r2.status_code
    }

@app.route('/async_get_data')
async def async_get_data():
    async with httpx.AsyncClient() as client:
        c1 = client.get('https://flash.siwalik.in/delay/1000/')
        c2 = client.get('https://flash.siwalik.in/delay/1000/')
        results = await asyncio.gather(c1, c2)

        return {
            'r1': results[0].status_code,
            'r2': results[1].status_code
        }

Both endpoints now make two GET requests to https://flash.siwalik.in/delay/1000/, which returns a simple response after one second. The first method get_data should look familiar to anyone who has used the Python requests library. r1 contains the result of the first API call, and r2 contains the result of the second API call. The method then returns the status code of each response.

The second method, async_get_data, looks a bit different, although the end result is the same. Going step by step, this is what is happening:

async with httpx.AsyncClient() as client:

The code creates an asynchronous context manager which makes the client object available. This is the same thing as a normal context manager, except that it allows the execution of asynchronous code. client is what makes the actual http calls.

c1 = client.get('https://flash.siwalik.in/delay/1000/')
c2 = client.get('https://flash.siwalik.in/delay/1000/')

Next, two variables are assigned the results of calling client.get() on the two API calls to the Flash API. At first you might assume that the results of the API calls will be stored in these variables, but actually, c1 and c2 are co-routines, not HTTP responses.

results = await asyncio.gather(c1, c2)

Now the magic of async happens. Here the return value of asyncio.gather()is assigned to the result variable, and it is awaited. When you see the await keyword, it means that the code will block execution there until the call to a co-routine is complete. The gather method itself is a co-routine, and will execute a sequence of other co-routines (like [c1, c2]) concurrently, and then return a list of results.

return {
    'r1': results[0].status_code,
    'r2': results[1].status_code
}

Finally, the method returns the status code for each HTTP response by accessing it within the array of results.

Calling get_data and async_get_data should result in the exact same result, but async_get_data will complete much faster. How much faster do you think it will finish?

Timing the Results

Now that you have an API with two endpoints that do the same thing, except one is async and one is not, you should return to using cURL to time them.

Start with get_data:

time curl "http://localhost:5000/get_data"
{"r1":200,"r2":200}

________________________________________________________
Executed in    3.56 secs      fish           external
   usr time    0.29 millis  295.00 micros    0.00 millis
   sys time    5.00 millis   48.00 micros    4.96 millis

You can see that both responses return an HTTP 200, and in total your endpoint took about 3.5 seconds to return. That makes sense: the external endpoints (Flash) paused for one second each and the extra 1.5 seconds of other overhead can be accounted for in DNS lookups, tcp connections, slow Comcast internet, and other internet related spaghetti.

Next, try timing the async_get_data endpoint:

```bash time curl “http://localhost:5000/async_get_data” {“r1”:200,”r2”:200}

Executed in 1.81 secs fish external usr time 6.21 millis 342.00 micros 5.87 millis sys time 0.06 millis 59.00 micros 0.00 millis ````

If in the previous section you guessed that the async version would be about twice as fast, you are correct! Why? Because the calls to the external API were run concurrently. That means that in this case, the entire act of retrieving the results from the Flash API was only as slow as the slowest call.

In fact, you can try adding a third call to each method. The first endpoint will take roughly 1.5 seconds longer, while the async version will still execute in roughly 1.8 seconds. You can keep adding HTTP calls to the aysnc version and it should continue to return in roughly the same amount of time until you start hitting various hardware, network and operating system level constraints.

A Real World Use Case.

You may think this is a contrived example. How often do you write endpoints that make multiple external HTTP calls? However, HTTP calls aren’t the only place where asyncio can make a difference. In fact, it’s right there in the name: Asynchronous Input/Output.

We often use databases to back our APIs and getting the results of a SQL query from a database server is often bound by I/O. We could replace one of the calls to the Flash API in our app with a call to a database. Let’s say this DB call returns a lot of data and takes 500 milliseconds to return. Imagine now that you replaced one of the HTTP calls in the endpoints written earlier with a call to a database. This seems like a more realistic example.

The first endpoint, get_data, would take roughly 1.5 seconds to return the result: 1 second for the HTTP call, and 0.5 seconds for the DB call.

The second endpoint, async_get_data, would take roughly 1 second to return the result: 1 second for the HTTP call, 0.5 seconds for the DB call, but both execute concurrently. This means it only takes as long as the slowest operation to return the result. That’s still 0.5 seconds we saved by using asyncio!

Keep in mind that if you need the results of one async call for the next async call, this won’t be much help.

Conclusion

In this article you learned how Python’s asyncio can speed up your application considerably in situations where your code is waiting on multiple instances of Input/Output. You also learned how asyncio can be used effectively and easily with the Flask web framework and the HTTPX library.

Asyncio won’t always make your API faster, but in certain situations like demonstrated in this tutorial, it can make a huge difference. Keep what you learned here in mind when writing APIs or other code in the future and you might gain some easy performance wins!

⚙ ⚙ ⚙

Austin Riba 24 Jun 2021 && [ code ] && 0 comments

I’ve been adding GNU Makefiles to all my projects recently and it’s not because I’ve suddenly become a C programmer.

Make was designed to be a build tool to make compiling complex programs with lots of source files easier. It does stuff like provide you with a way to describe dependencies between files and define tasks for compiling your program. It’s one of those legendary Unix programs that is still available on every Linux and Mac OS but most probably never use.

But I don’t use it for describing long chained build instructions. I just use it so I don’t have to remember commands.

I work on a larger set of projects now and they all do the same things, but just slightly different. A great example of this is starting up the dev server.

For Django:

python3 manage.py runserver

For Flask:

env FLASK_APP=src/api.py FLASK_ENV=development flask run

Even Docker:

docker run web -p8080:8080

Instead of trying to remember how to start each project, I’ve just started writing Makefiles. Now I have something like this for each project

run:
    python3 manage.py runserver

test:
    pyhton3 manage.py test

The other projects share the same command names. Now when I want to start a dev server for any project, I just cd to the directory and simply make run.

⚙ ⚙ ⚙

Austin Riba 29 Jan 2021 && [ code, python ] && 2 comments

If you’ve ever used Django, you might be familiar with Django Extensions Shell Plus. It allows you to execute $ ./manage.py shell_plus for a very handy iPython REPL with all your ORM models pre-imported. This snippet will allow us to accomplish the same with FastAPI or Flask.

The key is to use iPython’s embed feature to create the shell, and the SqlAlchemy class registry to auto import our models. Create a new file shell.py:

from IPython import embed

from app.database import Base

banner = 'Additional imports:\n'
from app.main import app
banner = f'{banner}from app.main import app\n'

for clzz in Base.registry._class_registry.values():
    if hasattr(clzz, '__tablename__'):
        globals()[clzz.__name__] = clzz
        import_string = f'from {clzz.__module__} import {clzz.__name__}\n'
        banner = banner + import_string

embed(colors='neutral', banner2=banner)

In this snippet, you will want to replace line 3 with the correct import path to your applications Base metadata class. I’ve also demonstrated how to add custom imports that might be handy: you’ll want to replace or remove lines 5-7 with the correct import path for your project’s “app” object.

Start it with python shell.py

And you should see something like this:

$ python shell.py
Python 3.9.1 (default, Jan  8 2021, 17:17:43)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

Additional imports:
from app.main import app
from app.sources.models import Source
from app.sources.models import Comment
from app.auth.models import User

In [1]:

In this example, the models Source, Comment, and User are available for me to use and query with normally.

⚙ ⚙ ⚙

Austin Riba 22 Jan 2021 && [ code, python, FastAPI ] && 3 comments

When developing a large database backed application, using an ORM (Object Relational Manager) can really benefit your project. There are quite a few ORMs for Python, but which work best with FastAPI?

You must be careful with considering which ORM to use. If your project relies heavily on interacting with a database you will end up writing a lot of code that relies on the ORM.

There are many ORMs that work with Python and they all have their strengths and weaknesses. If you are writing an application with FastAPI, there are constraints that need to be considered - mainly using an ORM that supports Python3 async.

ORMs Compared

TL;DR: use the following table to help you decide which ORMs might be worth looking in to.

SqlAlchemy

SqlAlchemy is probably the most well known ORMs for Python. The library is very established which makes it easy to find information online. It has over 17,000 questions on Stack Overflow. It also supports a wide variety of use cases and 3rd party integrations,.

Recently, the library’s author has been working a large version update. SqlAlchemy2 will have full Async support as well as improved syntax. This makes it a solid choice for use with FastAPI moving forward.

SqlAlchemy is not perfect however. Out of all the ORMs it is possibly the hardest to learn. The syntax is very verbose and the documentation is very difficult to navigate. Once you get past these hurdles though, you’ll be using a very powerful library.

Tortoise

Tortoise ORM is one of the new ORMs on the scene. It was designed from the beginning to fully take advantage of Python Aysnc, so it’s a great choice for use with frameworks like FastAPI.

Tortoise’s syntax also very closely mirrors that of the Django ORM meaning developers coming from Django will feel right at home. It’s concise syntax is also very easy to understand a learn.

Unfortunately because Tortoise has not been around long, it is missing some features. There is no support for queryable JSON fields, for example. This could be a deal breaker for some projects. However, if your needs are basic, Tortoise could be the best choice.

Pewee

Pewee Pewee is another mature ORM with a very clean and simple syntax. It also supports a variety of use cases, being fairly mature.

Unfortunately the project has no Aysnc support, making it not a great choice for Async frameworks. In fact author of the project actually appears to openly despise Python’s approach to Async and has shut down several attempts to add support to Pewee.

Pony

Pony is another Python ORM with a really unique syntax that appears to be a real joy to use. Whereas most ORMs either use manager objects or query builders, Pony attempts to keep your interaction with the database as close to plain Python as possible. Here is an example from the docs:

query = select(c for c in Customer
               if sum(o.total_price for o in c.orders) > 1000)

Beautiful!

Unfortunately, Pony does not have Async support, giving it the same problem as Pewee (though the maintainers don’t seem as vehemently opposed to it).

The other big ding against Pony is that it’s the only ORM on this list without a solution for database migrations. While not having a migrations system might be OK for small applications with 1 or 2 tables, as soon as your project grows and you need to be able to modify your schemas without destroying them in the process, a good migration system in essential.

Conclusion

If you are starting a new project with FastAPI or a similar framework and need an ORM, at this time I feel like there are really only 2 options.

If your project is using basic features of the database then Tortoise looks to be the winner. This is probably most projects.

If you know your project is going to need to take advantage of special or niche features of the database like queryable JSON or Gis fields, then you probably want to go with SqlAlchemy and take the hit on simplicity for flexibility.

⚙ ⚙ ⚙