Austin Riba 29 Jan 2021 && [ code, python ] && 2 comments

If you’ve ever used Django, you might be familiar with Django Extensions Shell Plus. It allows you to execute $ ./manage.py shell_plus for a very handy iPython REPL with all your ORM models pre-imported. This snippet will allow us to accomplish the same with FastAPI or Flask.

The key is to use iPython’s embed feature to create the shell, and the SqlAlchemy class registry to auto import our models. Create a new file shell.py:

from IPython import embed

from app.database import Base

banner = 'Additional imports:\n'
from app.main import app
banner = f'{banner}from app.main import app\n'

for clzz in Base.registry._class_registry.values():
    if hasattr(clzz, '__tablename__'):
        globals()[clzz.__name__] = clzz
        import_string = f'from {clzz.__module__} import {clzz.__name__}\n'
        banner = banner + import_string

embed(colors='neutral', banner2=banner)

In this snippet, you will want to replace line 3 with the correct import path to your applications Base metadata class. I’ve also demonstrated how to add custom imports that might be handy: you’ll want to replace or remove lines 5-7 with the correct import path for your project’s “app” object.

Start it with python shell.py

And you should see something like this:

$ python shell.py
Python 3.9.1 (default, Jan  8 2021, 17:17:43)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

Additional imports:
from app.main import app
from app.sources.models import Source
from app.sources.models import Comment
from app.auth.models import User

In [1]:

In this example, the models Source, Comment, and User are available for me to use and query with normally.

⚙ ⚙ ⚙

Austin Riba 22 Jan 2021 && [ code, python, FastAPI ] && 3 comments

When developing a large database backed application, using an ORM (Object Relational Manager) can really benefit your project. There are quite a few ORMs for Python, but which work best with FastAPI?

You must be careful with considering which ORM to use. If your project relies heavily on interacting with a database you will end up writing a lot of code that relies on the ORM.

There are many ORMs that work with Python and they all have their strengths and weaknesses. If you are writing an application with FastAPI, there are constraints that need to be considered - mainly using an ORM that supports Python3 async.

ORMs Compared

TL;DR: use the following table to help you decide which ORMs might be worth looking in to.

SqlAlchemy

SqlAlchemy is probably the most well known ORMs for Python. The library is very established which makes it easy to find information online. It has over 17,000 questions on Stack Overflow. It also supports a wide variety of use cases and 3rd party integrations,.

Recently, the library’s author has been working a large version update. SqlAlchemy2 will have full Async support as well as improved syntax. This makes it a solid choice for use with FastAPI moving forward.

SqlAlchemy is not perfect however. Out of all the ORMs it is possibly the hardest to learn. The syntax is very verbose and the documentation is very difficult to navigate. Once you get past these hurdles though, you’ll be using a very powerful library.

Tortoise

Tortoise ORM is one of the new ORMs on the scene. It was designed from the beginning to fully take advantage of Python Aysnc, so it’s a great choice for use with frameworks like FastAPI.

Tortoise’s syntax also very closely mirrors that of the Django ORM meaning developers coming from Django will feel right at home. It’s concise syntax is also very easy to understand a learn.

Unfortunately because Tortoise has not been around long, it is missing some features. There is no support for queryable JSON fields, for example. This could be a deal breaker for some projects. However, if your needs are basic, Tortoise could be the best choice.

Pewee

Pewee Pewee is another mature ORM with a very clean and simple syntax. It also supports a variety of use cases, being fairly mature.

Unfortunately the project has no Aysnc support, making it not a great choice for Async frameworks. In fact author of the project actually appears to openly despise Python’s approach to Async and has shut down several attempts to add support to Pewee.

Pony

Pony is another Python ORM with a really unique syntax that appears to be a real joy to use. Whereas most ORMs either use manager objects or query builders, Pony attempts to keep your interaction with the database as close to plain Python as possible. Here is an example from the docs:

query = select(c for c in Customer
               if sum(o.total_price for o in c.orders) > 1000)

Beautiful!

Unfortunately, Pony does not have Async support, giving it the same problem as Pewee (though the maintainers don’t seem as vehemently opposed to it).

The other big ding against Pony is that it’s the only ORM on this list without a solution for database migrations. While not having a migrations system might be OK for small applications with 1 or 2 tables, as soon as your project grows and you need to be able to modify your schemas without destroying them in the process, a good migration system in essential.

Conclusion

If you are starting a new project with FastAPI or a similar framework and need an ORM, at this time I feel like there are really only 2 options.

If your project is using basic features of the database then Tortoise looks to be the winner. This is probably most projects.

If you know your project is going to need to take advantage of special or niche features of the database like queryable JSON or Gis fields, then you probably want to go with SqlAlchemy and take the hit on simplicity for flexibility.

⚙ ⚙ ⚙

Austin Riba 28 Dec 2020 && [ Tutorial ] && 0 comments

Are you interested in adding geographical capabilities to your app? Perhaps you want to be able to search for nearby items on your site. Or maybe you want to know if your user’s location is within a specific region. With a few tools it is easy to add GIS (Geographical Information Systems) to your FastAPI back end.

Geometric Types and WKT (Well Known Text)

Programmers are used to data types like integers, strings and the like. But if we want to represent a location, shape, or line, how do we do that? Not only do we need to represent these types, we need to do it in a way that is interoperable with other tools.

One way to represent geometric types is by using Well Known Text format. Also known as WKT, this format provides an easy to read representation of geometries with widespread support, especially in open source tools. Here is how various geometries are represented in WKT:

As we can see, a POINT is represented simply by an X and Y coordinate. A LINESTRING is just a list of POINTs, a POLYGON is a special LINESTRING that starts and end at the same POINT. Multiple POLYGONs can be combined in a list (sometimes called a MULTI-POLYGON) to create complex shapes.

Geometry vs Geography

A geometry is a point, shape or line that exists on the Cartesian plane. The X and Y coordinates that make up a POINT are unit-less. But once we need to represent POINTs on earth (like locations) or LINESTRINGs (like roads) we need to use Geography.

Geographies are represented exactly the same in WKT, but can be treated differently by the software working with them. Typically, a Geography POINT uses longitude and latitude for X and Y, and these axis are limited to -90/90 degrees north, and 360 degrees east, respectively. Also calculations using geographies should be done on the surface of a sphere (defined by the SRID or UTM) instead of a flat plane.

The details here are not important. Just remember that if you are working with data that is meant to represent locations or earth (or space), which you probably are, you’ll want to use Geographies. Usually this just means using “Geography” instead of “Geometry” when typing out your queries and definitions.

PostGIS and Spatialite

Most databases need some kind of extension to work with Geometric data types. For Postgres, there is PostGIS. For Sqlite3, we have Spatialite.

Geoalchemy2

Models and Queries

⚙ ⚙ ⚙

Austin Riba 16 Dec 2020 && [ code, python, tutorial ] && 1 comments

Pydantic is one of the “secret sauces” that makes FastAPI such a powerful framework. The library is used everywhere in our projects: from validation, serialization and even configuration. Understanding better what Pydantic does and how to take advantage of it can help us write better APIs.

The problem of working with JSON

Before we delve into Pydantic, let’s quickly acknowledge the language modern APIs use: JSON. JSON is the lingua franca of modern APIs, and chances are any new app you start will speak it.

Let’s imagine an API that’s purpose is to provide a worldwide database of every pie imaginable: apple, pumpkin, even blackbird. This API allows us to search for pies based on name or ingredient and it also allows us to add pies. CRUD pies? Yes please.

OK, now imagine we want to add a pie to the database. We would send some JSON that looks something like this:

POST https://allpies.io/pies/

{
    "name": "API Pie",
    "calories": 9000,
    "description": "A tasty pie, clean presentation but a messy filling.",
    "ingredients": ["python", "pydantic", "FastAPI"]
}

Simple enough. Now let’s go over all the steps that our fictional Python backend would have to through in order to persist this delicious API Pie into the database.

The simplest implementation of an endpoint might look like this:

@app.post('/pies')
def create_pie(request):
    data = request.json()
    save_pie_to_database(data)

This obviously will not work. What if the JSON payload is missing fields, or is malformed? save_pie_to_database will definitely throw an error, and we want to avoid that. So let’s add a check to make sure all the required fields are supplied:

@app.post('/pies')
def create_pie(request):
    data = request.json()
    required_fields = ['name', 'calories', 'description', 'ingredients']
    if not set(required_fields).issubset(data.keys()):
      return HTTPError('Missing fields!')
    save_pie_to_database(data)

This is a little better, but still really bad. We don’t even tell the client which field they are missing if they fail to send one.

Even worse, we need to check to make sure the type of the data is correct. What if the client sends a string for calories? Saving to the database will fail. So we add another check:

@app.post('/pies')
def create_pie(request):
    data = request.json()
    required_fields = ['name', 'calories', 'description', 'ingredients']
    if not set(required_fields).issubset(data.keys()):
      return HTTPError('Missing fields!')
    if not type(data['calories']) == int:
      return HTTPError('Calories must be an integer!')
    save_pie_to_database(data)

This is just too ugly, and too much work to simply create a silly pie. This problem of taking arbitrary data and converting into Python and/or database objects is known as serialization. Making sure our data is good, is called validation. Pydantic helps us with both.

Serializers to the rescue

Most web frameworks provide some method of serializing/deserializing data from HTTP requests and responses. For example, Django Rest Framework dedicates an entire three chapters just to it’s serializers. They look like this:

class PieSerializer(serializers.Serializer):
    name = serializers.CharField(max_length=200)
    calories = serializers.IntegerField()
    description = serializers.CharField()
    ingredients = serializers.ListField(serializers.CharField())

We want to use FastAPI though, not Django. Luckily this is where Pydantic comes in.

Playing with Pydantic

If you haven’t already, install Pydantic into a virtualenv:

$ pip install pydantic

The following code snippets will run as valid Python, so fire up your editor and prepare to copy and paste!

Using Pydantic, let’s define a “model” (kinda like a serializer) for a pie. Then we will give it some data and see what happens!

from pydantic import BaseModel, constr
from typing import List

new_pie = {
    "name": "API Pie",
    "calories": 9000,
    "description": "A tasty pie, clean presentation but a messy filling.",
    "ingredients": [
        "python",
        "pydantic",
        "FastAPI",
    ]
}


class Pie(BaseModel):
    name: constr(max_length=200)
    description: str
    calories: int
    ingredients: List[str]


pie = Pie(**new_pie)
print(pie.name)
print(pie.calories)
print(pie.dict())

The output should look like this:

API Pie
9000
{'name': 'API Pie', 'description': 'A tasty pie, clean presentation but a messy filling.', 'calories': 9000, 'ingredients': ['python', 'pydantic', 'FastAPI']}

Neat! With just a few lines and some sweet Python3 type annotations we’ve created a way to take arbitrary data (in this case a dictionary, but we trust you can figure out how to use json.loads()) and turn it into a python object.

But what happens when the data is invalid? Add this to the bottom of the script:

from pydantic import ValidationError
new_pie['calories'] = 'Many, many calories. But not a number.'
try:
    pie = Pie(**new_pie)
except ValidationError as e:
    print(e.json())

The script will now output:

[
    {
        "loc": ["calories"],
        "msg": "value is not a valid integer",
        "type": "type_error.integer"
    }
]

Not only does Pydantic produce an error, but it gives us a nice error in JSON format that we can use how we see fit.

We can even add our own validators. Let’s ensure that the description of our pies always contains the word “delicious”, we don’t want the pie in our database otherwise:

from pydantic import BaseModel, constr, validator
from typing import List


class Pie(BaseModel):
    name: constr(max_length=200)
    description: str
    calories: int
    ingredients: List[str]

    @validator('description')
    def ensure_delicious(cls, v):
        if 'delicious' not in v:
            raise ValueError('We only accept delicious pies')
        return v

Now if we try to add our non-delicious pie, we get the following error:

[
    {
        "loc": ["description"],
        "msg": "We only accept delicious pies",
        "type": "value_error"
    }
]

Pydantic + FastAPI

Now that we have a basic understand of what Pydantic can do, we should be able to understand the functionality it brings to our FastAPI apps!

The following working FastAPI app has an endpoint that takes POST data and creates an entry into a fake pie database - if the data is a valid Pie Pydantic model, of course. Save the following code as app.py:

from fastapi import FastAPI
from pydantic import BaseModel, constr, validator
from typing import List
import uvicorn


class Pie(BaseModel):
    name: constr(max_length=200)
    description: str
    calories: int
    ingredients: List[str]

    @validator('description')
    def ensure_delicious(cls, v):
        if 'delicious' not in v:
            raise ValueError('We only accept delicious pies')
        return v


app = FastAPI()


def add_pie_to_database(pie: Pie) -> Pie:
    print(f'Adding {pie.name} to database!')
    return pie


@app.post('/pies/')
async def create_pie(pie: Pie):
    return add_pie_to_database(pie)

if __name__ == "__main__":
    uvicorn.run("app:app", host="127.0.0.1", port=5000, log_level="info")

Our Pie model is used here unchanged. Now check out line 29 in app.py. The route function create_pie takes a single parameter: pie, of type Pie. This tells FastAPI that this route should receive data that looks like a Pie.

Make sure you have FastAPI, Uvicorn, and our favorite command-line HTTP client, HTTPie installed:

pip install fastapi uvicorn httpie

Now you should be able to run the server with:

python app.py

Let’s try adding a pie (and having it sent right back to us) using HTTPie:

$ http POST http://127.0.0.1:5000/pies/ \
  name=APIPie \
  description="A delicious pie, clean presentation but a messy filling." \
  calories=900 \
  ingredients:='["python", "pydantic", "FastAPI"]'

We should see the following response in our terminal:

HTTP/1.1 200 OK
content-length: 151
content-type: application/json
date: Thu, 24 Dec 2020 05:40:38 GMT
server: uvicorn

{
    "calories": 900,
    "description": "A delicious pie, clean presentation but a messy filling.",
    "ingredients": [
        "python",
        "pydantic",
        "FastAPI"
    ],
    "name": "APIPie"
}

And if we try to add a not delicious pie?

$ http POST http://127.0.0.1:5000/pies/ \
  name=MudPie \
  description="This is actually just made of mud." \
  calories=unknown \
  ingredients:='["dirt", "water", "bark"]'

HTTP/1.1 422 Unprocessable Entity
content-length: 195
content-type: application/json
date: Thu, 24 Dec 2020 05:49:42 GMT
server: uvicorn

{
    "detail": [
        {
            "loc": [
                "body",
                "description"
            ],
            "msg": "We only accept delicious pies",
            "type": "value_error"
        },
        {
            "loc": [
                "body",
                "calories"
            ],
            "msg": "value is not a valid integer",
            "type": "type_error.integer"
        }
    ]
}

As expected, we get an HTTP error with a nice description of exactly what was wrong with our request.

Pydantic also helps us when we want to send JSON representations of pies to our users. Let’s add a method to get a fake Pie from our database and send it to the user:

from fastapi import FastAPI
from pydantic import BaseModel, constr, validator
from typing import List
import uvicorn


class Pie(BaseModel):
    name: constr(max_length=200)
    description: str
    calories: int
    ingredients: List[str]

    @validator('description')
    def ensure_delicious(cls, v):
        if 'delicious' not in v:
            raise ValueError('We only accept delicious pies')
        return v


app = FastAPI()


def add_pie_to_database(pie: Pie) -> Pie:
    print(f'Adding {pie.name} to database!')
    return pie


@app.post('/pies/')
async def create_pie(pie: Pie):
    return add_pie_to_database(pie)


def get_pie_from_database() -> Pie:
    return Pie(
        name="ApiPie",
        description="A delicious pie, clean presentation but a messy filling.",
        calories=9000,
        ingredients=[
            "python",
            "pydantic",
            "FastAPI"
        ],
    )


@app.get('/pie/')
async def get_pie():
    return get_pie_from_database()

if __name__ == "__main__":
    uvicorn.run("app:app", host="127.0.0.1", port=5000, log_level="info")

Let’s test the endpoint with a simple GET request:

http http://127.0.0.1:5000/pie/

And the response should be what you expect, a JSON representation of the pie we created in get_pie_from_database.

Conclusion

This was a simple introduction to Pydantic, but it should give you an idea of the functionality that Pydantic brings to FastAPI applications.

For additional information, check out the docs for Pydantic and some of the relevant sections of the FastAPI docs.

⚙ ⚙ ⚙

Austin Riba 11 Dec 2020 && [ code, python, tutorial ] && 1 comments

In this tutorial we will learn how to add database backed user authentication to our FastAPI application. Later is the series we will implement registration, password recovery, and more.

So you’re excited about FastAPI and you’ve been following the excellent documentation. At some point, you’ll come to the section on security which sets you up with a login view, some utilities for hashing passwords and a dependency injected current user object.

It works great! The only problem is now you are left with a working application, but your user database consists of a hardcoded dictionary. Obviously, this will not do for a real application.

In this tutorial, we will replace our fake users database dictionary with a real database backed user table. In the next part, we’ll add a registration endpoint so that people can sign up for accounts and login to your application.

Starting where you left off

If you haven’t already, go through the FastAPI documentation on security. We are going to pick up where it leaves off and you should be familiar with the concepts and code presented.

We should have an app.py that looks like this:

from datetime import datetime, timedelta
from typing import Optional

from fastapi import Depends, FastAPI, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
from jose import JWTError, jwt
from passlib.context import CryptContext
from pydantic import BaseModel

# to get a string like this run:
# openssl rand -hex 32
SECRET_KEY = "09d25e094faa6ca2556c818166b7a9563b93f7099f6f0f4caa6cf63b88e8d3e7"
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30


fake_users_db = {
    "johndoe": {
        "username": "johndoe",
        "full_name": "John Doe",
        "email": "johndoe@example.com",
        "hashed_password": "$2b$12$dQD2AD2Y.Aa8F3IliHPfk.yNESW7FZe3RmeT38K661sg/vds404ga",  # swordfish
        "disabled": False,
    }
}


class Token(BaseModel):
    access_token: str
    token_type: str


class TokenData(BaseModel):
    username: Optional[str] = None


class User(BaseModel):
    username: str
    email: Optional[str] = None
    full_name: Optional[str] = None
    disabled: Optional[bool] = None


class UserInDB(User):
    hashed_password: str


pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

app = FastAPI()


def verify_password(plain_password, hashed_password):
    return pwd_context.verify(plain_password, hashed_password)


def get_password_hash(password):
    return pwd_context.hash(password)


def get_user(db, username: str):
    if username in db:
        user_dict = db[username]
        return UserInDB(**user_dict)


def authenticate_user(fake_db, username: str, password: str):
    user = get_user(fake_db, username)
    if not user:
        return False
    if not verify_password(password, user.hashed_password):
        return False
    return user


def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
    to_encode = data.copy()
    if expires_delta:
        expire = datetime.utcnow() + expires_delta
    else:
        expire = datetime.utcnow() + timedelta(minutes=15)
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt


async def get_current_user(token: str = Depends(oauth2_scheme)):
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        username: str = payload.get("sub")
        if username is None:
            raise credentials_exception
        token_data = TokenData(username=username)
    except JWTError:
        raise credentials_exception
    user = get_user(fake_users_db, username=token_data.username)
    if user is None:
        raise credentials_exception
    return user


async def get_current_active_user(current_user: User = Depends(get_current_user)):
    if current_user.disabled:
        raise HTTPException(status_code=400, detail="Inactive user")
    return current_user


@app.post("/token", response_model=Token)
async def login_for_access_token(form_data: OAuth2PasswordRequestForm = Depends()):
    user = authenticate_user(fake_users_db, form_data.username, form_data.password)
    if not user:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Incorrect username or password",
            headers={"WWW-Authenticate": "Bearer"},
        )
    access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    access_token = create_access_token(
        data={"sub": user.username}, expires_delta=access_token_expires
    )
    return {"access_token": access_token, "token_type": "bearer"}


@app.get("/users/me/", response_model=User)
async def read_users_me(current_user: User = Depends(get_current_active_user)):
    return current_user

In case it’s been a while or you are starting from scratch, the minimum packages required to run this demo are:

pip install fastapi uvicorn passlib python-jose python-multipart bcrypt

And you can start the application with:

uvicorn app:app --reload

Now head over to the shiny auto-generated swagger docs at http://127.0.0.1:8000/docs and try it out. You should be able to click the “Authorize” button and login with the username and password:

username: johndoe \ password: swordfish

Just as you would expect from our fake_users_db.

Our goal now is to preserve this functionality while replacing fake_users_db with a real database.

Creating Gold with SqlAlchemy

For this example we are going to use SqlAlchemy ORM to interact with our database. There are a few ORMs out there, but SqlAlchemy is one of the more popular ones and just recently began supporting asynchronous io, so it’s perfect for use with FastApi. Install it:

pip install install sqlalchemy --pre

note: you can drop –pre if 1.4 is out of beta, which it might be by the time you read this.

To avoid adding to our already cluttered main.py file, we’re going to create a new module, database.py and set up SqlAlchemy there:

# database.py
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.orm import declarative_base
from sqlalchemy.ext.asyncio import AsyncSession

SQLALCHEMY_DATABASE_URL = "sqlite:///./sqlite3.db"

engine = create_async_engine(
    SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)

Base = declarative_base()


async def get_db():
    session = AsyncSession(engine)
    try:
        yield session
    finally:
        await session.close()

The first few statements define an engine (connection) to the database, as well as declaring an ORM model base for us to use (next step).

We also define a method to get a database session. This will be used in conjunction with FastAPI’s dependency injection system in order to provide access to the database where and when it is need.

We will also declare our User model, which will represent a user in the database:

from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.orm import declarative_base
from sqlalchemy import Boolean, Column, Integer, String
from sqlalchemy.ext.asyncio import AsyncSession

SQLALCHEMY_DATABASE_URL = "sqlite:///./sqlite3.db"

engine = create_async_engine(
    SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)

Base = declarative_base()


async def get_db():
    session = AsyncSession(engine)
    try:
        yield session
    finally:
        await session.close()


class User(Base):
    __tablename__ = 'users'

    id = Column(Integer, primary_key=True, index=True)
    username = Column(String, unique=True, index=True)
    email = Column(String, unique=True, index=True)
    full_name = Column(String, index=True)
    disabled = Column(Boolean, default=False)
    hashed_password = Column(String)

This is a typical Sqlalchemy declarative model. We’ve kept the structure the same as the users in our fake_users_db so that the changes in the rest of the application can remain minimal.

Speaking of changes in the main application, let’s get to the meat and potatoes. We will modify app.py with the following:

from datetime import datetime, timedelta
from typing import Optional

from fastapi import Depends, FastAPI, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
from jose import JWTError, jwt
from passlib.context import CryptContext
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import select

import database
# to get a string like this run:
# openssl rand -hex 32
SECRET_KEY = "09d25e094faa6ca2556c818166b7a9563b93f7099f6f0f4caa6cf63b88e8d3e7"
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30


class Token(BaseModel):
    access_token: str
    token_type: str


class TokenData(BaseModel):
    username: Optional[str] = None


class User(BaseModel):
    id: int
    username: str
    email: str
    full_name: str
    disabled: bool

    class Config:
        orm_mode = True


pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

app = FastAPI()


@app.on_event("startup")
async def start_db():
    async with database.engine.begin() as conn:
        await conn.run_sync(database.Base.metadata.create_all)


def verify_password(plain_password, hashed_password):
    return pwd_context.verify(plain_password, hashed_password)


def get_password_hash(password):
    return pwd_context.hash(password)


async def get_user(db: AsyncSession, username: str) -> database.User:
    result = await db.execute(select(database.User).filter_by(username=username))
    return result.scalars().first()


async def authenticate_user(db: AsyncSession, username: str, password: str) -> database.User:
    user = await get_user(db, username)
    if not user:
        return False
    if not verify_password(password, user.hashed_password):
        return False
    return user


def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
    to_encode = data.copy()
    if expires_delta:
        expire = datetime.utcnow() + expires_delta
    else:
        expire = datetime.utcnow() + timedelta(minutes=15)
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt


async def get_current_user(db: AsyncSession = Depends(database.get_db), token: str = Depends(oauth2_scheme)) -> database.User:
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        username: str = payload.get("sub")
        if username is None:
            raise credentials_exception
        token_data = TokenData(username=username)
    except JWTError:
        raise credentials_exception
    user = await get_user(db, username=token_data.username)
    if user is None:
        raise credentials_exception
    return user


async def get_current_active_user(current_user: User = Depends(get_current_user)) -> database.User:
    if current_user.disabled:
        raise HTTPException(status_code=400, detail="Inactive user")
    return current_user


@app.post("/token", response_model=Token)
async def login_for_access_token(db: AsyncSession = Depends(database.get_db), form_data: OAuth2PasswordRequestForm = Depends()):
    user = await authenticate_user(db, form_data.username, form_data.password)
    if not user:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Incorrect username or password",
            headers={"WWW-Authenticate": "Bearer"},
        )
    access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    access_token = create_access_token(
        data={"sub": user.username}, expires_delta=access_token_expires
    )
    return {"access_token": access_token, "token_type": "bearer"}


@app.get("/users/me/", response_model=User)
async def read_users_me(current_user: User = Depends(get_current_active_user)):
    return current_user

In the first changed block, we import a few things from Sqlalchemy that we will need, as well as import the database module we just defined.

We also modify the User Pydantic model. We want it to mirror the database representation so that it can correctly serialize data. Also notice the orm_mode = True line, that allows ORM objects (from sqlalchemy) to be passed in to Pydantic models (as we’ve defined here) and be correctly read and serialized.

At some point the database tables need to actually be created. A perfect time to do that would be when the app first starts up. So we use FastAPI’s startup lifecycle hook to tell Sqlalchemy to create the tables we defined with the declarative base.

The rest of the changes are to get_user(db: AsyncSession, username: str) and simple modifications to the other methods that rely on it. Instead of doing a dictionary access in fake_users_db we do an actual query on our database to look up a user by their username.

Because get_user requires a database connection, we perform a dependency injection in get_current_user as well as login_for_access_token. This ensures the database session is available everywhere that we need it.

Try it out

We now have a working application that functions pretty much the same as before, but will look up users in a Sqlite3 database instead of a dictionary. So how do we test it out? By inserting a user into the database of course!

First, make sure you are running your application. That will ensure the tables have been created (thanks to the start_db method we defined earlier).

$ uvicorn app:app --reload

Next, let’s add a user record to the generated users table.

From your command line, execute the following command:

$ sqlite3 sqlite3.db

This will open up a sqlite3 shell. From here, we can use SQL to add a record to the users table:

INSERT INTO users VALUES (1, 'johndoe', 'johndoe@example.com', 'John Doe', false, '$2b$12$dQD2AD2Y.Aa8F3IliHPfk.yNESW7FZe3RmeT38K661sg/vds404ga');

Notice the big long string at the end: it’s the same hashed password (“swordfish”) that we hardcoded into fake_users_db before!

Once you’ve created the record, you should be able to go back to the generated docs and login as you did before. Now try out the /users/me endpoint, it will return the data we inserted into the database!

Next steps

Manually adding users to your database is rarely what you want to do. In the next post in the series, we’ll implement a registration view so that users can use your API to request accounts.

⚙ ⚙ ⚙

Austin Riba 30 Sep 2020 && [ code, tutorials ] && 0 comments

Recently I did a live tutorial with CodingNomads on how to build an image sharing website (like Instagram) in Django. The demo is meant to show how quickly you can leverage Django to build and prototype web applications. You can view it on YouTube.

⚙ ⚙ ⚙

Austin Riba 14 Sep 2020 && [ other ] && 0 comments

Posting these here so I can look back in the future to remember when the forest fires started getting really bad.

The view at about 3:30 PM PST from the viewpoint near the Hwy 92/Skyline Blvd intersection.

A view of Nasa’s Worldview of the day

A video I took while driving on Highway 280, at 3:00 PM PST near Stanford, which was a dangerous thing to do and I do feel bad about it.

⚙ ⚙ ⚙

Austin Riba 3 Sep 2020 && [ birds ] && 4 comments

Could the simple California Towhee be one of the most underrated birds in North America?

First, let’s go over what makes this bird remarkably unremarkable to most people:

1. It’s drab, brown color even on males.
2. It’s conservation status is LC (Least Concern, they are doing fine).
3. Super common.
4. Boring call and song.

So by all outward appearances the California is a dull backyard bird that’s unlikely to warrant a second glance.

However, I have observed that the California Towhee has some unusual behavior that I have not seen in any other bird.

In my house we often leave a glass porch door open to let in some fresh air. Occasionally a bird will fly into our loft which inevitably causes them to do bird things like panic and fly in circles and into windows and such (by the way if this ever happens to you, the best way to remove a bird from your house is to throw a light towel or garment over them then bring them back outside). We have a lot of windows - our loft is a bit of a bird trap.

However, unlike Finches, Hummingbirds, etc I have never had to rescue a Towhee from the house. They are always able to find their way back out the door, and never fly into windows.

Not only do they not panic, but the Towhees often enter the house on purpose, while us humans stand by watching. They are exploratory creatures.

My wife and I once observed a particularly adventurous Towhee casually hop through the glass door to our loft, make it’s way down the stairs on the opposite side of the room (which are pretty complicated, floating stairs with a complete 180 degree turn), hang out in the kitchen a bit, then hop through another door into our bedroom. Once it was presumably satisfied that our house checked out, the bird simply retraced it’s steps out of the bedroom, up the stairs and out the porch door.

Another time I opened my eyes after a mid-day nap on our couch to see a Towhee on the window sill above my head, looking down at me, seemingly studying me. Once I moved it let out a little “chip” and simply flew out the open door.

To me, this shows that this bird has a particularly good sense of spatial awareness, which I can’t help but interpret as intelligence, especially when compared to other birds.

I’ve scoured the internet but I cannot find any confirmation or even mention of this strange behavior in Towhees. It does not seem like they are studied very often. Their Wikipedia page is particularly weak for such a common bird. Neither Audubon or Cornell seems to have much more than the usual ID type information.

I suspect this behavior might be a trait derived from them being a ground dwelling, foraging species. Perhaps being adept at navigating shrubs and underbrush translates well into navigating through… man made structures? Grasping at straws here, I am no ornithologist.

I would love to hear from anyone that might be able to provide any information about this fascinating bird!

⚙ ⚙ ⚙

Austin Riba 2 Sep 2020 && [ other ] && 0 comments

Here it comes again. One of my favorite questions.

“Wait, you live in Isla Vista?”

“Yes.”

“The college town by UCSB? You want to live there?”

“Yup.”

Most people’s idea of Isla Vista is either formed by having lived their in the college years, having known someone who lived there in the college years, or news stories about people in their college years who live there.

What is usually missed in all the stories about raucous parties and couch burnings is the fact that Isla Vista is located in one of the most naturally beautiful locations in all of California.

IV is a coastal town about 15 miles west of Santa Barbara. To the north is the rugged and expansive Los Padres National Forest. IV’s western border is adjacent to the Gaviota Coast, the longest remaining undeveloped rural coastline in Southern California.

You wouldn’t know it by looking at pictures of Deltopia or Halloween, but Isla Vista itself is rich in natural areas and parks. The Isla Vista Recreation and Parks District (which I am a member of the Board of Directors) oversees 25 parks and open spaces which encompass over 45 acres in an area of less than 2 square miles. And that doesn’t count the miles of coastline, county and state open naturalized open space, and the university’s natural preserve. All within walking distance for any resident.

That’s not to say Isla Vista is a total paradise. There are issues of density, lack of affordable housing, and a quickly eroding coastline. IV’s problems are to a large extent coastal California’s problems.

And then there is the student population. Are they loud and occasionally annoying? Yes. Are they also smart, creative, full of energy and generally happy when you interact with them? Absolutely. Given the choice, I’d take college kids as neighbors over aging NIMBY boomers 10 out of 10 times.

All of that was a long winded way to say that yes, we like it here. And we aren’t alone. While the larger population is transient, there is a core group of hippies, surfers, artists and professors that have chosen to make Isla Vista their permanent home.

OK, but what does any of this have to do with the time of Covid? Nothing really, except for an observation I’ve made about walking. Which is something a lot of us are doing more of now.

It’s no secret that walking is the absolute best way to become familiar with a place. But why? The low intensity exercise is stimulating, no doubt. But the real reason is that the speed you move through your environment while walking is perfectly aligned with the ability of your senses to take in and process information. Move too fast (as you do in a car or even cycling) and your sight becomes blurred, your sense of smell doesn’t have the time to pick up a lingering scent, sound is distorted or blocked by rushing wind or engine noise, and of course your are not actually touching the ground. Walking is the optimal state for all of the senses. It’s almost like we were made to do it.

Here is the silver lining of Covid times. To walk somewhere is to know it. To truly know something is to connect with and love that thing. Walking from your own home is one of the best ways to appreciate and love where you live in a way that, for example, driving to a place could never achieve. Over the last 6 months I’ve become so intimate with my direct surroundings that my feeling of “home” has expanded to the beach, the marsh, the fields and the trails I walk through them.

I feel lucky and privileged to live here.

⚙ ⚙ ⚙

Austin Riba 1 Sep 2020 && [ code ] && 1 comments

This website has been following the blog software hype train since it’s inception. The progression went like so:

1. Facebook “posts” (discontinued)
2. Blogger
3. Blosxom - a very early static site generator, way ahead of it’s time.
4. Wordpress
5. Jekyll
6. Hugo (current)

At the transition between Wordpress and Jekyll, like many others, I needed a solution for comments on a static site and Disqus was the clear choice. But then many of us learned that by using Disqus, we were allowing ads to be placed on our own pages. We were bogging down our websites with loads of third party trackers and possibly even violating our own reader’s privacy.

I started looking for alternatives recently (late, I know) and found Commento. It seems to cater to people who want to leave Disqus for the reasons I outlined above: privacy, performance and no ads.

Commento can be used as a service or it can be self hosted. The idea of self hosting the backend on my VPS was very appealing, so I gave it a shot. Overall, it was a pretty painless experience, especially when using the docker image. You simply spin it up with a postgres instance and set some config vars for things like email notifications, akismet integration, and google oauth. Then you place the script supplied from this backend on your page and it’s good to go.

Here are some links for people who would like to try self hosting Commento:

• Commento on GitLab
• The docs for self hosting
• Freecodecamp article on setting up 3rd party authentication (This is missing from the official docs)
• Note: The Disqus export takes a very long time, in my case 8 hours (!) for < 600 comments. It’s almost like they don’t want you to back up your data. I’d recommend queuing up a Disqus export first thing if you are planning to migrate off it.

Commento is very nice and I would not hesitate to pay for the hosted version (which I can always migrate to) if it weren’t for self hosting being interesting for me. You can check out the hosted service at commento.io.

⚙ ⚙ ⚙