← Go home

Rust and Serverless Blog Claps

June 07, 2020

Sometime back around February, I stumbled upon this Podcast Episode on Rust during one of my morning commutes to work, and was sold on learning Rust.

Why? At the time, I was a little burned out by React and JavaScript/TypeScript (what I used for work), so I figured learning a second language and more specifically, my first low-level language, was worth a shot.

The following weeks would feel like a rollercoaster in terms of my learning in Rust, computers, programming, and many other things like general motivation and self doubt.

Graph of Rust Knowledge

ℹ️ Note to self: Write a post on learning processes and methods (Rustlings, Leetcode, Exercism, side projects)

The Mission

After several weeks of dabbling with Rust “crates” (how Rust packages are referred to) like Actix (web framework), Diesel (ORM), and even making a MacOS system notification CLI tool, I arrived a point where I felt like I was not making anything fun.

Then, two days ago I came up with a miraculously fun idea, to recreate Medium’s “claps” feature.

Constraints

I had set some constraits for myself:

  • make this serverless (but don’t use Serverless)
  • don’t rely on a PaaS (like Vercel)
  • handle deployment and infrastructure myself

    • AWS API Gateway
    • AWS DynamoDB
    • AWS Lambda
  • write lambda execution code in Rust 🤪

Architecture

Going into this project, I had a vague idea of all the the moving parts, sort of a like a freshly opened Lego kit. It wasn’t until after I actually finished the feature that I went backwards a step to create this systems diagram.

Looking back, diagramming first would’ve been ideal but figuring things out on the fly also made this fun in an explorative way… I guess like building a Lego kit without the instruction manual 😮.

Systems Architecture Diagram

Essential Crates

I needed to use these crates to handle some key functionality of this feature.

serde

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically.

rusoto

The Rust equivalent of the AWS SDK for JavaScript. Unfortunately this only has an equivalent of AWS.DynamoDB, but not AWS.DynamoDB.DocumentClient (which exposes an API that accepts more idiomatic JavaScript to interface with DynamoDB).

lambda_runtime

This package makes it easy to run AWS Lambda Functions written in Rust.

This came with some crucial documentation for getting a custom Rust runtime into AWS Lambda. Currently, AWS only officially supports Node.js, Python, Ruby, Java, and Go runtimes.

Hurdles

AKA: hours of my life lost to the stackoverflow the void not caring to read and understand fundamental concepts thoroughly.

Local Development

In local development in VSCode, the Rust compiler helped catch so many type errors, and essentially prevented me from uploading Lambda function code that wouldn’t run. This was a huge plus thoroughout the development cycle.

I hadn’t (and still haven’t) figured out how to spin up a local Docker Rust runtime, so I ended up uploading new function code to Lambda and using the API Gateway test console, every time I needed to see the output of my Lambda execution code.

API Gateway console

JSON String to Struct

I think I lost about 18 hours to this - Saturday 8am to Sunday 2am. Initially, an API response error looked like a CORS issue (missing Access-Control-Allow-Origin header). This was only happening for a POST request, and not a GET request. Additionally I was definitely returning that header from Lambda code, so the problem must have been something else. I even did a sanity check by swapping out my Rust lambda function with a nearly identical Node.js function. No error. Ok…

I eventually narrowed it down to I was providing type annotations that pleased the compiler, but didn’t actually represent incoming data.

String to byte vector to struct

Confusing long story short, this is how you deserialize a JSON String into a typed Rust struct.

use serde_derive::{Deserialize, Serialize}; // 1.0.110;
use serde_json; // 1.0.53

// this is the data structure that I want to
// deserialize an incoming request payload into
#[derive(Serialize, Deserialize, Clone)]
struct Body {
    slug: String,
    claps: u32,
}

// this is the event object, coming from an
// API Gateway <> Lambda integration
struct Event {
    body: Option<String>, // <- troublesome request payload 👀
    #[serde(rename = "requestContext")]
    request_context: Option<RequestContext>,
}

// Inside Lambda handler
// async fn my_handler(e: Event) -> ...
    let body: Body = if let Some(json_string) = e.body {
        let byte_vector = json_string.into_bytes();
        serde_json::from_slice(&byte_vector).unwrap()
    } else {
        Body {
            claps: 0,
            slug: "init".to_string(),
        }
    };

    // access data on `body` to use in a Dynamo query
    format!("{}", body.claps);

Figuring this out resolved my unexpected errors when sending POST requests to the endpoint from my client-side code. GET requests didn’t have this issue since there was no JSON string payload to potentially deserialize incorrectly.

Success

At around Sunday morning at 2am, I got all my backend code to work (wow, a whole TWO functions 🤣😐)… I fist pumped the air in the dark, and went to bed.

Feature in action

I spent the next morning hooking up the client, or this blog (so meta!) to my Serverless, Rust Lambda Function powered API Gateway.

Gif of "claps" feature

… and getting carried away with CSS.

Learnings

  • How to deserialize a JSON string into a Rust Struct
  • How to create an “atomic counter” in DynamoDB from a potenially previously non-existent item
  • How to queue up “claps” and flush them in a debounced API call (React state and side effect management)
  • Medium uses GraphQL
  • Josh W. Comeau’s blog was a source of inspiration on a nearly identical feature. I believe he uses “Netlify Functions” (but one of my constraints was to not rely on a PaaS)

Next Steps

Spin up a local Docker container that simulates the AWS Rust lambda runtime.

Have that successfully interface with another local DynamoDB docker container.

Get into the habit of actually proofreading these posts.

🎉🎉🎉

Kevin Wang

👋 I'm Kevin Wang. Jazz GuitaristBaristaReceptionist Front End Engineer
This is my ever growing sandbox of sorts⛱.

Comments

Create Post

You must log in to comment.

Be the first to comment!