yyjhao

Free and Secure Messaging on Raspberry Pi with Telegram Bots

Sun, 25 Jul 2021 01:00:00 +0000

Telegram is a free and cross-platform instant messaging software. Telegram also offers a way to set up your own messaging bots. With Telegram bots, it’s easy to set up a tight interface with which you can talk to your home automation Raspberry Pi on the go without having to expose your whole Raspberry Pi as server.

P.S. Telegram bots can do a lot more , though we only focus on a small use case here.

Use Case

Push notification: the Raspberry Pi can sent a notification through a bot when home conditions change, e.g. door opened, or temperature dropped. The API will even allowed sending files like photos or videos, which make for a simple home surveillance system
Simple control interface: the Raspberry Pi can respond to a message sent to the bot and perform programmed actions, e.g. respond with the current condition at home

Why use telegram bots

It’s free! Both telegram the service/app and setting up a telegram bot is free. There are also virtually no concerns with rate limiting if you are only using the bot for yourself.
It’s (more) secure and therefore simpler. Instead of exposing a server to the internet and have to worry about defending your tiny server against all sorts of attacks, the communication strictly happens through the bot API via long polling.

Walkthrough

Setting up the bot

First of all, set up a telegram account if you haven’t done so.

Then, head over to Bot Father (yes, really) and send him (it?) a message /newbot. He will respond to help you set up the bot. Note down the token (and don’t share it with anyone!).

Quick start

First, let’s set up a quick project. Here we will use the node-telegram-bot-api project.

npm init
npm i node-telegram-bot-api
npm i -D typescript @types/node-telegram-bot-api ts-node
npx tsc --init

Create a new file, app.ts and start with

import TelegramBot from "node-telegram-bot-api";

const bot = new TelegramBot("<your access token from bot father", {
  polling: true,
});

bot.onText(/.*/, (msg, match) => {
  console.log("chat id", msg.chat.id);
  console.log("content", match![0]);
});

This basically logs every message that the bot receives.

Run this script with

`1`	`npx ts-node app.ts`

Now, try to start a conversation and send a message to your bot. I sent a simple ‘hello’. You should see something like this

node-telegram-bot-api deprecated Automatic enabling of cancellation of promises is deprecated.
In the future, you will have to enable it yourself.
See https://github.com/yagop/node-telegram-bot-api/issues/319. node:internal/modules/cjs/loader:1092:14
chat id 1234
content /start
chat id 1234
content hello

For the rest of this post, I will use 1234 as the chat ID. Substitute with your own.

Note down the chat id - it refers to the chat that you have started with the bot. We can use this as an authorization mechanism by limiting the bot to only respond to our chat. Let’s update the script to

bot.onText(/.*/, (msg, match) => {
  console.log("chat id", msg.chat.id);
  if (msg.chat.id !== 1234) {
    console.log("Ignoring message as chat ID mismatched");
    return;
  }
  console.log("content", match![0]);
});

You can test this out by asking your friend to try to contact the bot, or use a different telegram account. You should see something like this:

chat id 2345
Ignoring message as chat ID mismatched

Someone can still compromise telegram or your telegram account and thus gain access to your bot, but at least it’s easier to defend against that. In the case that they do gain access, they will also only able to perform actions that you have programmed the bot to support.

Sending messages

We will also use this chat ID as the ‘channel’ to send any push notifications. As a demo, let’s set up a simple pinging notification.

const chatId = 1234;

setInterval(() => {
  bot.sendMessage(chatId, `ping ${new Date().toString()}`);
}, 5000);

Responding to messages

We can also program the bot to respond to messages. Put this below the first onText code snippet:

bot.onText(/^What is ([a-z\s]+)\?/, (msg, match) => {
  console.log("chat id", msg.chat.id);
  if (msg.chat.id !== chatId) {
    console.log("Ignoring message as chat ID mismatched");
    return;
  }
  if (match![1] === "the time") {
    bot.sendMessage(chatId, `The time is ${new Date().toTimeString()}`);
  } else {
    bot.sendMessage(chatId, `I don't know what ${match![1]} is`);
  }
});

This sets up a very simple ‘AI’. It should be easy to extend this to perform arbitrary actions based on the messages that the bot receives.

Complete code:

import TelegramBot from "node-telegram-bot-api";

const bot = new TelegramBot("<access token from bot father>", {
  polling: true,
});

const chatId = 1234; // get your chat ID from the log

bot.onText(/^What is ([a-z\s]+)\?/, (msg, match) => {
  console.log("chat id", msg.chat.id);
  if (msg.chat.id !== chatId) {
    console.log("Ignoring message as chat ID mismatched");
    return;
  }
  if (match![1] === "the time") {
    bot.sendMessage(chatId, `The time is ${new Date().toTimeString()}`);
  } else {
    bot.sendMessage(chatId, `I don't know what ${match![1]} is`);
  }
});

bot.onText(/.*/, (msg, match) => {
  console.log("chat id", msg.chat.id);
  if (msg.chat.id !== chatId) {
    console.log("Ignoring message as chat ID mismatched");
    return;
  }
  console.log("content", match![0]);
});

setInterval(() => {
  bot.sendMessage(chatId, `ping ${new Date().toString()}`);
}, 5000);

Set up rust + wasm + TypeScript without webpack

Sun, 25 Apr 2021 01:00:00 +0000

1 Set up the rust code and wasm-pack

install wasm-pack
Set up a new project with wasm-pack. Run

1
2

$ wasm-pack new hello-wasm
$ cd hello-wasm

Build with this command

`1`	`$ wasm-pack build --target web`

This should produce a pkg/ directory which should contain some d.ts file, a .wasm file and a .js file in ES6 module format.

2 Set up the TypeScript code

Now we should set up a JavaScript project.

Run npm init to create an npm project.
Run npm i -D typescript to install tsc
Run npx tsc --init to create a tsconfig.json file
For cleaner organization of our source code, let’s say all ts files should be put in a specific ts/ directory
Edit the tsconfig.json file. Here are the field values we should make sure to update

{
  "module": "es2015", // must be es2015 or later
  "outDir": "./build" // other directory works too, but we need a outdir
}

For our little demo purpose, we will call the demo greet() function from the generated rust project.

Let’s make a file ts/entry.ts, with the following content:

import init, { greet } from "../pkg/hello_wasm.js";
init().then(function () {
  greet();
});

This mostly follows the official guide .

It’s a bit tricky here because TypeScript won’t prevent you from calling greet() without calling init() first, but doing so will lead to a crash. This is something to keep in mind, though it’s not specific to our setup.

Next we verify that this builds:

`1`	`$ npx tsc -p .`

3 Set up a development iteration process

Now let’s set up an actual html file.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Wasm Demo</title>
  </head>
  <body>
    <script type="module" src="/build/ts/entry.js"></script>
  </body>
</html>

Next, we can use any quick static server iterate on our code. I will use live-server here as an example:

Run

1
2

$ npm i -D live-server
$ npx live-server

On another terminal, run

`1`	`$ npx tsc --watch -p .`

Now making any changes in the typescript should trigger a rebuild and an autoreload.

Unfortunately there isn’t a watch mode for wasm-pack yet as of April 2021 (see this issue for some discussion).

4 Set up a production build process

Most of the important outputs should be in /build and /pkg. For deployment, it’s pretty easy to use any standard bundling tool to build a single bundle out of the generated entry.js file. There are two other small details to take care of

The html file assumes the build layout - we can fix this by replacing the src attribute in the html file with sed or other fancy tools as part of our build process
The generated .js file from wasm-pack assumes that the wasm file will be adjacent. We can either ensure that this happens, or pass in the path to the wasm file when calling the init function.

Set Up A Raspberry Pi As An Auto-refreshing Web Dashboard

Sat, 03 Apr 2021 01:00:00 +0000

Set up a Raspberry Pi to run Chromium in kiosk mode

credit: This section is mostly based on https://die-antwort.eu/techblog/2017-12-setup-raspberry-pi-for-kiosk-mode/ with some tweak to work with the auto-refresh section later.

Set up

First, start with a fresh install of Raspbian Lite - we don’t want all the bloat because we are only using the Raspberry Pi to display a dashboard

Then, we perform the usual Raspberry Pi setups, including things like updating password or enabling SSH.

Importantly, run sudo raspi-config, in “Boot Options”, Select “Desktop / CLI” and then “Console Autologin”. We will auto-run startx later in bash.

Now, install the dependencies - X server, a window manager (we use the lightweight openbox ), Chromium and NodeJS.

1
2

$ curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
$ sudo apt install --no-install-recommends xserver-xorg x11-xserver-utils xinit openbox chromium-browser nodejs

A quick note here about apt - if you are running commands directly in your terminal, apt offers a much better experience than apt-get!

Configuration

Edit /etc/xdg/openbox/autostart and replace its content with the following:

# Disable any form of screen saver / screen blanking / power management
xset s off
xset s noblank
xset -dpms

We disable screen blanking and power management which doesn’t make sense for our dashboard.

Next, we want to automatically start the X server on login. We can modify the .bash_profile to include this

`1`	`[[ -z $DISPLAY && $XDG_VTNR -eq 1 ]] && startx -- -nocursor`

Note that we have configure the Pi to log into command line above. Combining with this change it will cause the Pi to start an empty X session on boot.

Now that we are done, we will implement a service to (re)start Chromium on demand.

Create a service to (re)start Chromium

In my experience, running Chromium for a prolonged period of time usually results in the classic ‘aw snap’ crash. A solution here is to restart the browser from time to time, usually at night when the dashboard is not needed.

Also, to iterate changes on the dashboard, it’s better to have some mechanism to restart the browser without having to attach a keyboard to the dashboard display.

Lastly, when we push a new change to the Pi, it’s possible that new code changes can crash the dashboard page, so, we want a way to refresh that doesn’t rely on logics in the web page.

To achieve these, we can implement a service in node.js that provides an end point to restart the Chromium browser. In addition, we will launch Chromium in the ‘kiosk mode’:

The kiosk mode allows chromium to take over the whole screen with only the webpage and no frames or browsers UIs.

Here’s a simple, one-file service in TypeScript with minimum dependencies:

import { ChildProcess, spawn } from "child_process";
import * as http from "http";

// A global state to keep track of the current chromium process
let curProcess: ChildProcess | null = null;

function stop() {
  if (curProcess !== null) {
    process.kill(-curProcess.pid);
  }
  curProcess = null;
}

function restart(url: string) {
  stop();
  curProcess = spawn(
    "chromium-browser",
    [
      // This disables the annoying 'chrome is out of date' window
      "--check-for-update-interval=604800",
      "--kiosk",
      "--incognito",
      url,
    ],
    { detached: true }
  );
}

// Not the prettiest HTTP service implementation...
http
  .createServer(function (request, response) {
    function fail(error?: Error) {
      response.writeHead(400, { "Content-Type": "application/json" });
      response.end(JSON.stringify({ success: false, error }));
    }

    function succeed() {
      const result = { success: true };
      response.writeHead(200, { "Content-Type": "application/json" });
      response.end(JSON.stringify(result), "utf-8");
    }

    if (request.method !== "POST") {
      fail();
    }

    const restartMatch = /\/restart\??(.*)/.exec(request.url!);
    if (restartMatch) {
      const url = unescape(restartMatch[1]);
      restart(url);
      succeed();
    } else {
      fail();
      return;
    }
  })
  .listen(3001);
console.log("Server running at http://127.0.0.1:3001/");

You can also use the JavaScript code directly.

We use an HTTP server, but we can also use websocket or whatever else that’s convenient for the use case.

Lastly, we can set up a new service to make sure that this service starts running when we reboot the Pi. To do so, we create a new systemd service. For example, browser_config.service in /etc/systemd/system:

[Unit]
Description=BrowserControl

[Service]
# We need this to make sure that chromium is opened in the right X server session
Environment="DISPLAY=:0"
Environment="XAUTHORITY=/home/pi/.Xauthority"
WorkingDirectory=/path/to/compiled/js/
ExecStart=/usr/bin/node /path/to/compiled/js/file_name.js
Restart=always
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=browser-control-service
User=pi
Group=pi

[Install]
WantedBy=graphical.target

Then, we make sure to start this service on boot:

1
2
3

$ sudo systemctl daemon-reload
$ sudo systemctl start browser_control_service
$ sudo systemctl enable browser_control_service

Finally, to restart the sole chromium browser, your app or script can post to http://localhost:3001/restart, e.g.

`1`	`$ curl -X POST http://localhost:3001/restart?http://example.com`

Roll Your Own CSS-in-JS Library (5) - Tagged Templates

Wed, 24 Mar 2021 01:00:00 +0000

In this post, we will explore a different CSS-in-JS syntax.

CSS-in-JS but using CSS syntax

End result:

Notice that we get to write CSS similar to template strings rather than as objects.

This has a few advantages:

Our CSS code now looks closer to vanilla CSS.
We can sequence CSS properties.
We can duplicate CSS properties to support fallback behavior.

In practice, 1 is probably the strongest driver for adopting this syntax, as typically there’s no benefit in sequencing CSS properties except for fallback behavior. However, even with the object syntax, one can support fallback behavior by adding a post-processor to the rule generation pipeline.

What is styled``

ES6 tagged template

mdn has more information. We will provide a quick summary here.

ES6 introduces a new string interpolation syntax aside from template srings. The “`” can be immediately preceded by some function name. For example styled`abc ${someVar} de ${someMoreVar} fg`. This is treated similar to styled(["abc ", " de ", " fg"], someVar, someMoreVar). Note that styled need not return a string.

As a quick example to get us more familiar with this syntax, let’s implement this using a similar approach in our second post .

function styled(strings: string, ...expressions: string[]) {
    const dynamicFactory = styleDef.css(strings, ...expressions);
    const result: any = React.forwardRef((props: T, ref) => {
        return React.createElement("div", {
            ...props,
            className: [...css(props), (<any>props).className ?? ""].join(" ")
            ref
        });
    });
    return result;
}

Notice that we delegated the class name creation to a different function similar to our previous approach. Let’s implement our new StyleDef. This will end up pretty similar to what we have before, except that instead of generating stylesheets from an object, we can concatenate the strings instead.

class StyleDef {
  #selectorCounter = 0;
  #sheet: CSSStyleSheet;
  #prefix: string;

  constructor(prefix: string) {
    const sheetElement = document.createElement("style");
    document.head.appendChild(sheetElement);
    this.#sheet = sheetElement.sheet!;
    this.#prefix = prefix;
  }

  style<Props>(
    strings: string[],
    ...expressions: (props: Props) => string[]
  ): (props: Props) => string {
    const memo = new Map<string, string>();
    return (props) => {
      const cssBody = strings
        .reduce((accu, cur, index) => {
          accu.push(cur);
          if (index < expressions.length) {
            accu.push(expressions[index](props));
          }
        }, [])
        .join("");
      if (memo.has(cssBody)) {
        return memo.get(cssBody)!;
      }
      const selector = this.#generateSelector();
      memo.set(cssBody, selector);
      const ruleString = `.${selector}: ${cssBody}`;
      this.#sheet.insertRule(ruleString, this.#sheet.cssRules.length);
      return selector;
    };
  }

  #generateSelector = () => {
    return `${this.#prefix}-${this.#selectorCounter++}`;
  };
}

Notice that this simple approach is missing an important feature - nested CSS selectors. To support this, we will have to implement some kind parsers, which we will do in the next section.

CSS variables again?

We may also want to consider implementing this with a similar approach that uses CSS variables like in our third post . This is actually a lot harder. Consider the following code snippet:

const Block = styled`
    border: 1px solid black;
    color: ${(props) => props.color};
    height: ${(props) => props.height}px;
    ${(props) => (props.hasWidth ? "width: 100px;" : "")}
`;

If we simply replace the functions with the variables like what we did before, it will only work for the color property. height will look like this

height: var(--some-var)px;

which is not valid.

We can fix the issue with height if we can somehow detect that there’s some residue string before and after a dynamic css value, and update the function to include it. In the above example, we will transform ${props => props.height}px into ${props => props.height + "px"}.

However, width simply won’t work at all because a css variable can’t be a property-value pair. We have to disallow patterns like this.

Since we plan to handle nested style, let’s write a parser and fix both issues along the way.

High level approach

We will first parse the template strings interpolated with functions into an AST. Then, we can walk through the AST, collect CSS rules and convert the nested selectors into proper CSS selectors. While we are collecting CSS rules, we can look at strings before and after functions and note them as ‘partial CSS values’, which allows us to transform the function as we discussed above.

Defining the syntax

With the introduction of nested selectors, we will need to update our ‘CSS’ syntax a bit. We will allow not just CSS declaration (property: value;) in a block (wrapped by { and }), we will also allow selectors which can open their own block. i.e. the following is valid

.classname {
  height: 1px;
  .descendant {
    height: 1px;
    .descendant-2 {
      height: 1px;
    }
  }
}

Note that when generating CSS rules we will always wrap the definition in some class name so our underlying syntax doesn’t need to allow declaration in the global scope, i.e. the following is not valid to our parser

height: 1px;

… because we will convert it to

1
2
3

.generated-class {
  height: 1px;
}

Our AST then should consist of the follow nodes:

nestable selector node
declaration (property-value pair)

…where a nestable selector node can contain other nestable selector nodes or declarations.

With the addition of string templates, it’s more interesting. Now we need to split out the declaration node, because it be either

property, followed by value, or
property, followed a partial value start, then a function, then a partial value end.

Visitor pattern

Recall the visitor pattern we touched on in the previous post. Based on our discussion above, we can define a visitor function like so:

type VisitorState<T> =
  | {
      type:
        | "partial_value_start"
        | "partial_value_end"
        | "nest"
        | "property"
        | "value";
      value: string;
    }
  | {
      type: "close";
    }
  | {
      type: "dynamic_function";
      value: (props: T) => string;
    };

interface NestedCssTemplateVisitor<T, Output> {
  onVisit(state: VisitorState<T>): void;
  done(): Output;
}

Note the addition of "close" which is necessary to represent that we have exited a nested selector.

Parsing and visiting

Note that the visitor pattern doesn’t require us to supply a tree directly. We can also parse the string and call the visitor along the way. Following this, we will define 2 functions. The main one is walkNestedCssInJs, which will take the ES6 tagged template and a visitor, then parse the the former and call the visitor accordingly.

The other one is advanceNestedCssInJs which parses some string from the tagged template and extracts the relevant information. We use the word advance here because it’s advancing our state machine - for example, if we are in the nest state, we expect to see either property or nest next, and if we are in the property state, we expect to see value next.

// We can define this based on VisitorState, but
// we spell it out here for clarity
type ParseState =
  | "partial_value_start"
  | "partial_value_end"
  | "nest"
  | "close"
  | "property"
  | "value";

function advanceNestedCssInJs(
  currentState: WalkState,
  cssInJsString: string
): {
  nextState: WalkState;
  value: string;
} {
  // we will work on this later
}

function walkNestedCssInJs<T, Output>({
  initialState,
  strings,
  expressions,
  visitor,
}: {
  // Require an initial state here so that we can call
  // the visitor to supply a root selector before calling
  // this.
  initialState: ParseState;
  strings: TemplateStringsArray;
  expressions: ((props: T) => string)[];
  visitor: NestedCssTemplateVisitor<T, Output>;
}) {
  let state: ParseState = initialState;
  for (let i = 0; i < strings.length; i++) {
    const s = strings[i];
    let currentState: WalkState = {
      state,
      index: 0,
    };
    while (true) {
      const result = advanceNestedCssInJs(currentState, s);
      currentState = result.nextState;
      visitor.onVisit({
        type: currentState.state,
        value: result.value,
      });
      if (currentState.index === s.length) {
        break;
      }
    }
    state = currentState.state;
    if (i !== strings.length - 1) {
      visitor.onVisit({
        type: "dynamic_function",
        value: expressions[i],
      });
    }
  }
}

Next let’s implement advanceNestedCssInJs. To do so, we will first need to define a few regexes.

Here’s our quick and hacky regexes for the various states:

close: /(})/
property: /([a-z-]+):/
value: /[^"'{};\s]*(?:["'].*["'])*(?:url$.*$)*[^"{}';]*;/
selector: /([^;{}"\s()]+[^;{}"(]*(?:(?:not)|(?:.*-[a-z]+)\([^;{}"(]*)*)[\s]*{/

However, due to the existence of ‘partial values’, it’s possible that the regex may not match. As such, we split out the detection into 2 states - when we have advance through a property, we know that the next token must be a value or partial_value_start. If the regex fail to match value we can assume that the token is partial_value_start.

This lead us to the following algorithm:

const reSelectorOrPropertyOrClose = /^\s*(?:([^;{}"\s()]+[^;{}"(]*(?:(?:not)|(?:.*-[a-z]+)\([^;{}"(]*)*)[\s]*{)|(?:([a-z-]+):)|(})\s*/i;
const reValue = /^\s*([^"'{};\s]*(?:["'].*["'])*(?:url\(.*\))*[^"{}';]*;)\s*/i;
const rePartialValueEnd = /([^;]*);\s*/i;

function advanceNestedCssInJs(
  currentState: WalkState,
  cssInJsString: string
): {
  nextState: WalkState;
  value: string;
} {
  switch (currentState.state) {
    case "partial_value_start":
      const nextSemiColonMatch = rePartialValueEnd.exec(cssInJsString);
      if (!nextSemiColonMatch) {
        throw new Error("no match");
      }
      return {
        nextState: {
          state: "partial_value_end",
          index: nextSemiColonMatch.index + nextSemiColonMatch[0].length,
        },
        value: nextSemiColonMatch[1],
      };
    case "close":
    case "nest":
    case "value":
    case "partial_value_end": {
      const match = reSelectorOrPropertyOrClose.exec(
        cssInJsString.substring(currentState.index)
      );
      if (!match) {
        throw new Error("no match");
      }
      if (match[1]) {
        return {
          nextState: {
            state: "nest",
            index: match.index + match[0].length + currentState.index,
          },
          value: match[1],
        };
      } else if (match[2]) {
        return {
          nextState: {
            state: "property",
            index: match.index + match[0].length + currentState.index,
          },
          value: match[2],
        };
      } else if (match[3]) {
        return {
          nextState: {
            state: "close",
            index: match.index + match[0].length + currentState.index,
          },
          value: match[3],
        };
      } else {
        throw new Error("Unexpected");
      }
    }
    case "property": {
      reValue.lastIndex = currentState.index;
      const match = reValue.exec(cssInJsString.substring(currentState.index));
      if (!match) {
        return {
          nextState: {
            state: "partial_value_start",
            index: cssInJsString.length,
          },
          value: cssInJsString.substring(currentState.index).trim(),
        };
      }
      return {
        nextState: {
          state: "value",
          index: match.index + match[0].length + currentState.index,
        },
        value: match[1],
      };
    }
    default:
      throw new Error("unexpected");
  }
}

Converting to flat CSS with CSS variables

Next we will implement a visitor that converts the nested template CSS into a flat, standard CSS string.

To convert nested selectors, we maintain a stack as we walk through the tree.

To transform the dynamic function, we keep track of partial_value_start and partial_value_end so that we can perform the following transformation:

1
2
3

const transformed = (props: Props) => {
  return partialValueLeft + lastDynamicFunction(props) + partialValueRight;
};

Here’s the full visitor source code:

class NestedCssTemplateToFlatCssVisitor<Props>
  implements
    NestedCssTemplateVisitor<
      Props,
      {
        rules: string[];
        variablesValueMapping: (
          props: Props
        ) => {
          [variableName: string]: string;
        };
      }
    > {
  #ruleStack: Array<{ selector: string; declarations: string[] }> = [];
  #rules: string[] = [];
  #variablesFuncMapping: {
    [variableName: string]: (props: Props) => string;
  } = {};
  #partialValueLeft: string = "";
  #lastDynamicFunction: ((props: Props) => string) | null = null;

  constructor(public readonly generateVariableName: () => string) {}

  onVisit(state: VisitorState<Props>) {
    switch (state.type) {
      case "partial_value_start":
        this.#partialValueLeft = state.value;
        break;
      case "partial_value_end":
        const variableName = this.generateVariableName();
        const lastDynamicFunction = this.#lastDynamicFunction;
        this.#lastDynamicFunction = null;
        if (!lastDynamicFunction) {
          throw new Error("unexpected");
        }
        const partialValueLeft = this.#partialValueLeft;
        this.#partialValueLeft = "";
        const partialValueRight = state.value;
        this.#variablesFuncMapping[variableName] = (props: Props) => {
          return (
            partialValueLeft + lastDynamicFunction(props) + partialValueRight
          );
        };

        this.#currentRule().declarations.push(`var(${variableName});`);
        break;
      case "nest":
        const selector =
          this.#ruleStack.length === 0
            ? state.value
            : joinSelectors(this.#currentRule().selector, state.value);
        this.#ruleStack.push({
          selector,
          declarations: [],
        });
        break;
      case "close":
        const lastRule = this.#ruleStack.pop();
        if (!lastRule) {
          throw new Error("unexpected");
        }
        this.#rules.push(
          `${lastRule.selector} { ${lastRule.declarations.join(" ")} }`
        );
        break;
      case "property":
        this.#currentRule().declarations.push(`${state.value}:`);
        break;
      case "value":
        this.#currentRule().declarations.push(`${state.value};`);
        break;
      case "dynamic_function":
        this.#lastDynamicFunction = state.value;
        break;
      default:
        const _never: never = state;
    }
  }
  done() {
    return {
      rules: this.#rules,
      variablesValueMapping: (props: Props) => {
        const result: any = {};
        Object.keys(this.#variablesFuncMapping).forEach((v) => {
          result[v] = this.#variablesFuncMapping[v](props);
        });
        return result;
      },
    };
  }

  #currentRule = () => {
    if (this.#ruleStack.length === 0) {
      throw new Error("unexpected");
    }
    return this.#ruleStack[this.#ruleStack.length - 1];
  };
}

Wrapping up

The last part here is again creating our StyleDef and styled function.

class StyleDef {
  #selectorCounter = 0;
  #sheet: CSSStyleSheet;
  #prefix: string;

  constructor(prefix: string) {
    const sheetElement = document.createElement("style");
    document.head.appendChild(sheetElement);
    this.#sheet = sheetElement.sheet!;
    this.#prefix = prefix;
  }

  style<Props = {}>(
    strings: TemplateStringsArray,
    ...expressions: ((props: Props) => string)[]
  ): {
    className: string;
    variablesValueMapping: (props: Props) => CSS.PropertiesHyphen;
  } {
    const selector = this.#generateSelector();
    let counter = 0;
    const visitor = new NestedCssTemplateToFlatCssVisitor(
      () => `--${selector}-${counter++}`
    );

    visitor.onVisit({
      type: "nest",
      value: `.${selector}`,
    });
    walkNestedCssInJs({
      initialState: "nest",
      strings,
      expressions,
      visitor,
    });
    visitor.onVisit({
      type: "close",
    });
    const { rules, variablesValueMapping } = visitor.done();

    rules.forEach((ruleString) =>
      this.#sheet.insertRule(ruleString, this.#sheet.cssRules.length)
    );
    return {
      className: selector,
      variablesValueMapping,
    };
  }

  #generateSelector = () => {
    return `${this.#prefix}-${this.#selectorCounter++}`;
  };
}

const styleDef = new StyleDef("prefix");

export function styled<Props extends {}>(
  strings: TemplateStringsArray,
  ...expressions: ((props: Props) => string)[]
) {
  const { className, variablesValueMapping } = styleDef.style(
    strings,
    ...expressions
  );
  const result: any = React.forwardRef((props: Props, ref) => {
    return React.createElement("div", {
      ...props,
      className: [className, (<any>props).className ?? ""].join(" "),
      style: {
        ...variablesValueMapping(props),
        ...(<any>props).style,
      },
      ref,
    });
  });
  return result;
}

Build

One issue with this syntax is that unless we can ditch the advance syntax like nested CSS, we are incurring even more overhead due to the parser. However, this is the kind of work that’s perfect to be moved to build time.

The approach to convert the code in post into a build-time compiler is very similar to what we have in the previous post , so we won’t go though it again.

Roll Your Own CSS-in-JS Library (4) - Static Extraction

Mon, 15 Mar 2021 01:00:00 +0000

Static extraction is HARD

Firstly, the set-up of static extraction is more complicated - there needs to be a build process, and the static extractor needs to fit into that build process.

More importantly, static extraction will require some form of static evaluation. Take the following code for example:

import getColor from "helpers";

const Block = styled({
  color: getColor("block"),
});

In order to statically generate the css styles, the extractor needs to look at at least one other file called “helpers”. In this case, it may also need to understand what “helpers” is pointing to - is it to some top level file called “helpers”, or a module? Moreover, “helpers” may itself import getColor from yet another file. Even worse, Due to the flexibility of JavaScript, getColor may also be defined in many interesting ways.

A perfect extractor will need to somehow evaluate all of the code and generate the class name as well as the CSS code.

At the same time, we should only extract the minimal amount of code necessary, both due to performance concerns and the general desire to avoid side effects during a build process.

This is really hard to do right. Facebook prepack may offer a glimpse of hope, but it’s still very early in the development stage and the development seems to have stalled as of March 2021.

For now, most build-time CSS-in-JS libraries impose some kind of restrictions to make things easier, and most non-build-time CSS-in-JS libraries avoid building any static extraction at all to stay flexible (see arguments from styled-component and emotion ).

Nevertheless, let’s give it a try and build a simple static extractor for our little library.

Building a simple static extractor

In our last post we have built a dynamic CSS-in-JS library powered by CSS variables. We will create a script that generate CSS code and replaces the call to styled with some static class names.

To keep things simple, we will eschew a lot of important features like source mapping and integration with build tools. We will also use JavaScript this time to make it easier to run in the terminal.

Basic idea

We will impose some restrictions here. In particular:

All expressions in the function parameter of styled must be static.
- This means that all static values needs to be inline constants.
We will replace all function calls with styled(), i.e. styled doesn’t need to be imported, but developers also can’t redefined styled to be something else

Then we will build a simple script that reads through only a single js file, extracts these definitions, replaces them with a definition with some constant class name and generates a corresponding stylesheet.

Since we are dealing with a single JS file at a time, our implementation can be encapsulated in the following function:

const process = (fileContent) => {
  //...
  return {
    jsCode: "updated js code",
    cssCode: "generated css code",
  };
};

Parsing JavaScript code

Our first step is to parse the source code into some AST so that we can meaningfully analyze it. To do so, we will use babel .

const babelParser = require("@babel/parser");
const process = (fileContent) => {
  const ast = babelParser.parse(fileContent, {
    sourceType: "module",
    plugins: ["jsx"],
  });
};

That’s pretty easy. We specify sourceType: "module", because we want to support generate import and export statement. jsx is also necessary to support the JSX syntax.

At this point, it may be useful to print out the AST to get a sense of what it looks like.

Notice that the AST consists of ’nodes’, each has a type attribute that describes what type of node it is. Each node can nest other nodes. For example, the VariableDeclaration node has a declarations list, where some elements can be VariableDeclarators, that specify the id (e.g. x in const x =), and the init part of the declarator (e.g. the expression to call a function).

Identify valid `styled` calls

Next, we want to traverse the AST, look for our styled() call, generate some CSS code based on that, and replace the styled() call with something that only reference the class name. To do so, we use the traverse function from babel. This function follows the visitor pattern .

We will call traverse with the AST with a ‘visitor’, and the traverse function walks through the AST and calls the proper functions in the visitor when it encounters specific nodes. In this case, we only care about the ‘call expression’, since we want to know when styled is called.

const babelParser = require("@babel/parser");
const { default: traverse } = require("@babel/traverse");
const process = (fileContent) => {
  const ast = babelParser.parse(fileContent, {
    sourceType: "module",
    plugins: ["jsx"],
  });
  traverse(ast, {
      CallExpression(path) {
        if (
          path.node.callee.name === "styled" &&
          path.node.arguments.length === 1 &&
          checkArgument(path.node.arguments[0])
        ) {
          // We know that this is something like styled({}).
        }
    });
};

Notice the checkArgument call - remember when we said we want to impose some restriction? This is where we check against the first restriction - all expressions must be static.

function checkArgument(node) {
  // Make sure that the node is an object
  if (node.type === "ObjectExpression") {
    return node.properties.every((node) => {
      // The object has to consists of only key: value pairs,
      // i.e. no spread syntax
      return (
        node.type === "ObjectProperty" &&
        // The value can be either string or function
        (node.value.type === "StringLiteral" ||
          node.value.type === "ArrowFunctionExpression") &&
        // The key can be either an identifier (e.g. a: "b"),
        // or a string literal (e.g. "a": "b")
        (node.key.type === "Identifier" || p.key.type === "StringLiteral")
      );
    });
  }
}

Statically evaluate each `styled` call

Now that we know that the call is valid, let’s statically evaluate it. Our restriction has made things much simpler - we can pretty much evaluate the object as it is, with functions represented by some ’tokens’.

We will need to pull out our nestedDeclarationToRuleStrings function again. Notice that in nestedDeclarationToRuleStrings we don’t ever try to evaluate the functions, so we can update the implementation and replace functions with our AST nodes instead. We annotate the changes with comments below:

function nestedDeclarationToRuleStrings(rootClassName, declaration) {
  const resultRules = [];
  const resultMapping = {};

  let count = 0;
  function _makeNewVariable(valueDef) {
    const variableName = `--${rootClassName}-${count}`;
    count++;
    resultMapping[variableName] = valueDef;

    return variableName;
  }

  function _helper(selector, declaration) {
    const nestedNames = [];
    const cssProps = {};

    for (let prop in declaration) {
      const value = declaration[prop];

      // We use to check if `typeof value !== "Function"`, now we check
      // if value.type is defined. This is a bit hacky but should work
      // as long as <type> tag and a `type` css property don't exist.
      if (value instanceof Object && !value.type) {
        nestedNames.push(prop);
      } else {
        cssProps[prop] = value;
      }
    }

    const lines = [];
    lines.push(`${selector} {`);
    for (let prop in cssProps) {
      // Similarly We use to check if `typeof value !== "Function"`, now
      // we check if value.type is defined.
      if (cssProps[prop].type) {
        lines.push(`${prop}:var(${_makeNewVariable(cssProps[prop])});`);
      } else {
        lines.push(`${prop}:${cssProps[prop]};`);
      }
    }
    lines.push("}");

    resultRules.push(lines.join("\n"));

    nestedNames.forEach((nestedSelector) =>
      _helper(
        joinSelectors(selector, nestedSelector),
        declaration[nestedSelector]
      )
    );
  }
  _helper("." + rootClassName, declaration);
  return {
    rules: resultRules,
    // We use to return a function (props) => inline style, now
    // we just return a mapping from CSS variable names to the AST nodes
    // representing the original functions
    variablesValueMapping: resultMapping,
  };
}

Running this function should generate a list of CSS rules and a map from the CSS variable to the AST nodes representing each function in the original code.

const process = (fileContent) => {
  const ast = babelParser.parse(fileContent, {
    sourceType: "module",
    plugins: ["jsx"],
  });
  let counter = 0;
  let cssCode = "";
  traverse(ast, {
    CallExpression(path) {
      if (
        path.node.callee.name === "styled" &&
        path.node.arguments.length === 1 &&
        checkArgument(path.node.arguments[0])
      ) {
        const className = "prefix-" + counter;
        counter++;
        const { rules, variablesValueMapping } = nestedDeclarationToRuleStrings(
          className,
          convertToDeclaration(path.node.arguments[0])
        );
        cssCode += rules + "\n";
        const code = `
React.forwardRef((props, ref) => {
  return React.createElement("div", {
    ...props,
    className: ["${className}", props.className ?? ""].join(" "),
    style: {
      ${
        // somehow generate the style code here
      }
    }
      ...props.style
    },
    ref
  });
})
`;
        // replace the call expression with our new code
        path.replaceWith(babelParser.parseExpression(code));
      }
    },
  });
};

Using the nestedDeclarationToRuleStrings function that we have just built, coupled with our counter-base class name function and a simple string to collect css code, we were able to statically generate the class name and the css rules without actually running the JavaScript code in the string.

We then plug the statically evaluated class name into to the code string, which is the same the implementation for styled, parse it into an AST and replace the node representing the styled call with it.

Generating the transformed JavaScript code

First we need to fill in the style attribute in our generated React code. We will use @babel/generator to generate code from AST - think of it as the inverse of @babel/parser.

The variablesValueMapping maps a variable name, which is a string, to a node representing a function. We will first convert this node to some code as a string, then insert the string to our generated React code. We will do that for every key-value pair.

This is what it looks like:

const { default: generate } = require("@babel/generator");
`
    style: {
${Object.keys(variablesValueMapping)
  .map((k) => {
    return `"${k}": (${generate(variablesValueMapping[k]).code})(props),`;
  })
  .join("\n")}
      ...props.style
    },
`;

Finally, the output JavaScript code can be generated by generate(ast).code.

Complete source code

Below is the complete source code with an example JavaScript code input, copied from our previous example. Click run to check out the output CSS and JS strings.

Last words

While we simplified our extractor to make it almost not useful, our code generation is probably on the more complicated side. Most tools will try to generate only the class name and the list of variables to avoid being too specific to a framework.

Also, depending on where we specify the dynamic logics, we may not need CSS variables at all. Facebook’s StyleX is such an example. The API allows for only static styles with ‘variants’, and only generates class names. This makes static codegen simpler and removes the potential overhead of CSS variables.

Roll Your Own CSS-in-JS Library (3) - Using CSS Variables

Sat, 13 Mar 2021 01:00:00 +0000

In this post we will discard our changes in the previous post and switch to a new approach.

End result

CSS variables

CSS variables are also known as CSS Custom Properties. At a very high level:

We can set some value on a variable using --variable-name: value. The variable name has to start with --, and the definition has to happen within some selector or as inline styles.
We can then read the value of a variable and supply it to some CSS property with the var() function, e.g. color: var(--theme-color);
CSS Variables will cascade into inner elements.

Approach

We will update our CSS rule generation code to support dynamic properties with CSS variables, we then set the value of each variable using inline styles. This way, we will generate only a single class name per dynamic style.

Take this styled component we built in the last post as an example:

const Block = styled((props: Props) => ({
  border: "1px solid black",
  background: props.mode === "light" ? "white" : "black",
  fontSize: props.size === "big" ? "2em" : "1em",
}));

We should generate a CSS rule that looks like this

.c1 {
  border: 1px solid black;
  background: var(--c1-background);
  font-size: var(--c1-font-size);
}

then we need to apply the following inline style:

<div
  style={{
    "--c1-background": props.mode === "light" ? "white" : "black",
    "--c1-font-size": props.size === "big" ? "2em" : "1em",
  }}
></div>

Update to `CssLikeObject`

With this approach, we need to know exactly what properties are dynamic. Our previous API for dynamic styles doesn’t work any more. Let’s update our CssLikeObject definition:

type DynamicCssProperties<Props = {}> = {
  [k in keyof CSS.PropertiesHyphen]:
    | CSS.PropertiesHyphen[k]
    | ((props: Props) => CSS.PropertiesHyphen[k]);
};

type CssLikeObject<Props = {}> =
  | {
      [selector: string]: DynamicCssProperties<Props>;
    }
  | DynamicCssProperties<Props>;

With this new API, our styled component now looks like this:

const Block = styled({
  border: "1px solid black",
  background: (props: Props) => (props.mode === "light" ? "white" : "black"),
  fontSize: (props: Props) => (props.size === "big" ? "2em" : "1em"),
});

Rebuilding `nestedDeclarationToRuleStrings`

First, let’s define the new behavior. nestedDeclarationToRuleStrings should take in a rootClassName and a new CssLikeObject as before. However, it’s not sufficient to just output some CSS rules - we also need to know what CSS variables are created and how those CSS variables should be filled. We can do that with this new function interface:

function nestedDeclarationToRuleStrings<Props = {}>(
  rootClassName: string,
  declaration: CssLikeObject<Props>
): {
  rules: string[];
  variablesValueMapping: (props: Props) => CSS.PropertiesHyphen;
};

In addition to the original rules string, we also provide a way to compute the values of all variables that the rules string may contain. We will place the rules strings into our dynamic stylesheets and the rootClassName to the class name of our elements as before, but now we also need to compute the values of variables and place them into the inline styles of our elements.

Before doing that, let’s update the implementation of nestedDeclarationToRuleStrings

function nestedDeclarationToRuleStrings<Props = {}>(
  rootClassName: string,
  declaration: CssLikeObject<Props>
): {
  rules: string[];
  variablesValueMapping: (props: Props) => CSS.PropertiesHyphen;
} {
  const resultRules: string[] = [];
  const resultMapping: {
    [s: string]: (props: Props) => string;
  } = {};

  // Make a new CSS variable and record the mapping
  let count = 0;
  function _makeNewVariable(valueDef: (props: Props) => string) {
    const variableName = `--${rootClassName}-${count}`;
    count++;
    resultMapping[variableName] = valueDef;

    return variableName;
  }

  function _helper(selector: string, declaration: CssLikeObject<Props>) {
    // We split the props into either nested dynamic css rules
    // or plain dynamic css props.
    const nestedNames: string[] = [];
    const cssProps: DynamicCssProperties<Props> = {};

    for (let prop in declaration) {
      const value = (<any>declaration)[prop];

      if (value instanceof Object && typeof value !== "function") {
        nestedNames.push(prop);
      } else {
        (<any>cssProps)[prop] = value;
      }
    }

    const lines: string[] = [];
    // Collect all generated css rules.
    lines.push(`${selector} {`);
    for (let prop in cssProps) {
      // collect the top level css rules
      if (typeof (<any>cssProps)[prop] === "function") {
        // Since this is a function, we know that it should be represented
        // by css variables
        lines.push(`${prop}:var(${_makeNewVariable((<any>cssProps)[prop])});`);
      } else {
        lines.push(`${prop}:${(<any>cssProps)[prop]};`);
      }
    }
    lines.push("}");

    // Each string has to be a complete rule, not just a single
    // property
    resultRules.push(lines.join("\n"));

    // Go through all nested css rules, generate string css rules
    // and collect them
    nestedNames.forEach((nestedSelector) =>
      _helper(
        joinSelectors(selector, nestedSelector),
        (<any>declaration)[nestedSelector]
      )
    );
  }
  _helper("." + rootClassName, declaration);
  return {
    rules: resultRules,
    variablesValueMapping: (props) => {
      const result: { [s: string]: string } = {};
      Object.keys(resultMapping).forEach((k) => {
        result[k] = resultMapping[k](props);
      });
      return result;
    },
  };
}

Now we can update our StyleDef class. Notice that there’s no longer a need to differentiate static and dynamic styles, so we will remove the staticStyle function and replace it with a more generic style function. All other utilities remain the same.

class StyleDef {
  #selectorCounter = 0;
  #sheet: CSSStyleSheet;
  #prefix: string;

  constructor(prefix: string) {
    const sheetElement = document.createElement("style");
    document.head.appendChild(sheetElement);
    this.#sheet = sheetElement.sheet!;
    this.#prefix = prefix;
  }

  style<Props = {}>(
    declaration: CssLikeObject<Props>
  ): {
    className: string;
    variablesValueMapping: (props: Props) => CSS.PropertiesHyphen;
  } {
    const selector = this.#generateSelector();
    const { rules, variablesValueMapping } = nestedDeclarationToRuleStrings(
      selector,
      declaration
    );
    rules.forEach((ruleString) =>
      this.#sheet.insertRule(ruleString, this.#sheet.cssRules.length)
    );
    return {
      className: selector,
      variablesValueMapping,
    };
  }

  #generateSelector = () => {
    return `${this.#prefix}-${this.#selectorCounter++}`;
  };
}

Next let’s create a similar styled function

const styleDef = new StyleDef("somePrefix");

function styled<T extends {}>(
  declaration: CssLikeObject<T;
) {
  const {
     className,
     variablesValueMapping
  } = styleDef.style(declaration);
  const result: any = React.forwardRef((props: T, ref) => {
      return React.createElement("div", {
          ...props,
          className: [className, (<any>props).className ?? ""].join(" "),
          style: {
             ...variablesValueMapping(props),
             ...(<any>props).style
          },
          ref
      });
  });
  return result;
}

This is very similar to what we have before, but it works just fine if we pass in a static style. In addition, if we take a look at the few issues we mentioned in the last post, there are a few benefits.

Since we generate only a single class per style:

There’s no worry with an unbounded number of class names (duh!)
Descendant selector will work
Static extraction (generating CSS code during build time) is doable

Issues

Does not support IE

IE doesn’t support CSS variables . Although if you still need to support IE in 2021, supporting CSS-in-JS will be the least of the troubles.

Large generated stylesheet

The approach detailed in this post doesn’t try to avoid generating duplicated styles across different style definitions. However, we can still apply the same atomic CSS strategy to this approach, so this is not really an inherent issue with the approach of using CSS variables.

Restricted syntax

This may not be a big deal, but we had to switch our API to be able to detect dynamic rules. This means that we can’t do something like this:

const Block = styled((props: Props) =>
  props.isByWidth
    ? {
        width: "100px",
      }
    : {
        height: "100px",
      }
);

Again we can achieve the same result with either API, but to some, the above may be a bit cleaner.

Performance overhead?

As with anything fancy, CSS variables probably come with some overhead. This post mentions the fact that CSS variable can slow down startup time, while another post talks about some gotchas with scoping. Some more proper investigation is likely warranted though, as it’s possible that the issue in the first post was due to the scoping issue mentioned in the second.

More importantly, because it’s a lot easier to perform static extraction with this approach, we can likely reduce the performance footprint compared to our previous approach. We will talk about static extraction in our next post.

Roll Your Own CSS-in-JS Library (2) - Dynamic Styles

Fri, 12 Mar 2021 01:00:00 +0000

In our last post we developed a simple CSS-in-JS library, but it was missing an important feature - supporting dynamic styles. We will patch that in this post.

End result

Dynamic styles are more useful when we are building components. In this post, we will use React as our component library. Here’s how our library may be used in production:

Dynamic class names

This is the simplest approach that requires making the smallest change to our little library. Our approach will be to generate the class name based on the props being passed, then supply that to our react component. This means that Block under the hood should look like this:

const classNameGenerator = styleDef.dynamicStyle({
  border: "1px solid black",
  background: props.mode === "light" ? "white" : "black",
  fontSize: props.size === "big" ? "2em" : "1em",
});
const Block = (props: Props) => {
  return <div className={classNameGenerator(props)}>{props.children}</div>;
};

A simple implementation based on what we did previously may look like this

class StyleDef {
  dynamicStyle<Props>(
    declaration: (props: Props) => CssLikeObject
  ): (props: Props) => string {
    return (props) => {
      return this.staticStyle(declaration(props));
    };
  }

  //...
}

Done? Well, this works, but there’s a big issue here - every time staticStyle is executed, a new css class name and a set of corresponding rules are created! We can fix this by doing a bit of memoization.

class StyleDef {
  dynamicStyle<Props>(
    declaration: (props: Props) => CssLikeObject
  ): (props: Props) => string {
    // Makes from css rule to class name
    const memo = new Map<string, string>();
    return (props) => {
      // JSON.stringify is not deterministic, but good enough
      // for our demo.
      const hash = JSON.stringify(props);
      if (memo.has(hash)) {
        return memo.get(hash)!;
      }
      const result = this.staticStyle(declaration(props));
      memo.set(hash, result);
      return result;
    };
  }
  //...other code
}

This fixes the performance issue we talked about above and is probably a good enough MVP, but there are still other issues.

Issues

Unbounded number of class names

In our toy example there can only be 4 possible sets of styles. But it’s easy to construct something that can have an infinite number of possible props:

1
2
3

styleDef.dynamicStyle((left: number) => {
  left;
});

One solution is to use inline styles for these properties.

No more descendant selectors

With staticStyle we can do something like this

const someClass = styleDef.staticStyle({
  color: "green",
});

const otherClass = styleDef.staticStyle({
  ":hover": {
    [someClass]: {
      color: "red",
    },
  },
});

This is not trivial with dynamicStyle. One solution here is to use CSS variables, like this:

const someClass = styleDef.staticStyle({
  color: "var(--someClassColor)",
});

const otherClass = styleDef.dynamicStyles((props: { isRed: boolean }) => ({
  "--someClassColor": "green",
  ":hover": {
    "--someClassColor": props.isRed ? "red" : "blue",
  },
}));

Generating CSS code during build time is hard

Unless we can iterate through all possible values of props at build time, we won’t be able to generate all CSS rules. We can also settle with some ‘default props’ and just generate that single one, but that’s not ideal.

Large generated stylesheet

To some extend this can be an issue with our previous static-only library as well, but having dynamic styles makes the issue more pronounced. In our example, even though border stays the same, each new class will repeat this property. We can split out the static styles into its own staticStyle call, but that’s a bit awkward.

One solution here is to use the atomic css pattern, detailed below.

Optimization - Atomic CSS

What we are doing doesn’t exactly match atomic css , but the end result is similar.

The apporach is to programmatically break down each ‘class’ into multiple classes, each with only one css property. Then, we will similarly memoize the class names for each rules. This way, if a CSS property-value is repeated across multiple style definitions, all of those styles will share the same class name.

For example, suppose we render our example Block twice, once with { mode: "light"; size: "big" } and once with { mode: "dark"; size: "big" }. This will create the following css rules:

.c1 {
  border: 1px solid black;
}

.c2 {
  background: white;
}

.c3 {
  font-size: 2em;
}

.c4 {
  background: black;
}

We will then pass to the first call the following classnames: "c1 c2 c3", and to the second call: "c1 c4 c3".

Implementation

Firstly, we will write a helper to split out each style declaration into multiple atomic declarations.

function isolateDeclaration(
  declaration: CssLikeObject,
  prefix: string[] = []
): CssLikeObject[] {
  return Object.keys(declaration)
    .sort()
    .flatMap((key) => {
      const value = (<any>declaration)[key];

      if (value instanceof Object) {
        return isolateDeclaration((<any>declaration)[key], [...prefix, key]);
      } else {
        // Expand the prefix array, e.g. if prefix = ['a', 'b'],
        // we return {a: {b: declaration}}
        return prefix.reduceRight(
          (result, cur) => {
            return {
              [cur]: result,
            };
          },
          {
            [key]: declaration[key],
          }
        );
      }
    });
}

And example output:

> isolateDeclaration({
...     ":hover": {
.....         color: "green"
.....     },
...     color: "red"
... })
[ { ':hover': { color: 'green' } }, { color: 'red' } ]

Next we will update our staticStyle and dynamicStyle method.

class StyleDef {
  #memo = new Map<string, string>();
  staticStyle(declaration: CssLikeObject): string[] {
    return isolateDeclaration(declaration).map((d) => {
      const hash = JSON.stringify(d);
      if (this.#memo.has(hash)) {
        return this.#memo.get(hash);
      }
      const selector = this.#generateSelector();
      const ruleStrings = nestedDeclarationToRuleStrings(selector, declaration);
      ruleStrings.forEach((ruleString) =>
        this.#sheet.insertRule(ruleString, this.#sheet.cssRules.length)
      );
      this.#memo.set(hash, selector);
      return selector;
    });
  }

  dynamicStyle<Props>(
    declaration: (props: Props) => CssLikeObject
  ): (props: Props) => string[] {
    return (props) => {
      return this.staticStyle(declaration(props));
    };
  }
  //...
}

We updated the function interface to return an array instead of a single string, because we will now most likely return multiple class names. Moreover, we move the memoization into the class state since we want each staticStyle call to share the memoization.

Interestingly, we also reverted our optimization for dynamicStyle since memoization is now done at the staticStyle level, so we no longer need to worry about creating a new rule per call.

React factory

Finally let’s wrap this up by implementing what we have promised at the start of this post - a React factory to create styled components.

const styleDef = new StyleDef("somePrefix");

function styled<T extends {}>(
  declaration: (props: T) => CssLikeObject
) {
  const dynamicFactory = styleDef.dynamicStyle(declaration);
  const result: any = React.forwardRef((props: T, ref) => {
      return React.createElement("div", {
          ...props,
          className: [...dynamicFactory(props), (<any>props).className ?? ""].join(" ")
          ref
      });
  });
  return result;
}

Are dynamic styles really necessary?

Probably not. After all, without CSS-in-JS, the usual pattern is to have some classes defined in css files, then reference them in the UI code. We can replicate that easily by generating these classes with static styles. This allows for writing CSS code in the same file/dependency tree as the JavaScript code, and the dynamic-ness is handled separately by the JavaScript UI logic instead of encapsulated in the CSS-in-JS magic. That being said, dynamic styles does provide convenience and some sort of elegance (depending on who you ask).

Other ideas

We haven’t really dealt with the other issues mentioned above other than the issue of generating a large stylesheet. Our API for the react component is also a bit awkward: if we intend the component to only have static styles, we have to pass something like () => CssLikeObject. We can work around this by trying to allow both (T) => CssLikeObject and CssLikeObject, but can we come up with something more elegant than that?

In our next post , we will resolve all of these issues by switching to a different approach - using CSS variables.

Roll Your Own CSS-in-JS Library (1) - Introduction

Sun, 07 Mar 2021 01:00:00 +0000

CSS-in-JS is an idea to tackle various issues with CSS by writing them in JavaScript (or whatever else that can compile to JavaScript, such as TypeScript). Some popular examples include libraries like styled component , emotion and JSS .

I originally started this series to document some of my experiments with CSS-in-JS, but it later turned out to be a fun investigation project. In total, this series will cover

The basic idea of run-time static CSS-in-JS (this post)
The basic idea of run-time dynamic CSS-in-JS
Run-time dynamic CSS-in-JS with CSS variables
An extremely simplified approach to build-time CSS-in-JS
A simple approach to implementing tagged template based CSS-in-JS

Please note that this series focus on the high level ideas with a simple demo, so expect to notice a lot of missing features. In particular, there’s no mention of at-rules or & parent selectors. Adding them shouldn’t be very hard, but will likely make this series even longer.

The basic idea of run-time static CSS-in-JS

Credit: some code here is inspired by nano-css .

End Result

We will build a simple library that allows us to do this:

We will expand this to support dynamic styles in the next post . For now, the library is only limited to static styles.

Using the CSSStyleSheet API

This is a relatively obscure DOM API that allows programmatic manipulation of CSS rules. More details can be found on mdn .

Here’s a high level overview of how to use this API for our purpose:

Firstly, we can create a <style> tag:

1
2

const sheetElement = document.createElement("style");
document.head.appendChild(sheetElement);

The DOM API allows us to access the stylesheet object representing the styles in the <style> tag by sheetElement.sheet.

The stylesheet object allows us to manipulate the css rules, e.g. we can insert a rule by:

`1`	`sheetElement.sheet.insertRule(ruleString, sheet.cssRules.length);`

With this API, we can implement a CSS-in-JS library like so:

Generate a class name, then
Attach the rules to that class name, then
Assign the class name to some DOM element

Example implementation

Defining our in-JS representation

Firstly, let’s define the type of CSS objects. Here we want the ability to specify CSS rules with a nestable object representation so that we can specify rules with selectors. At the same time we want to be able to not specify the root class name in the object and automatically generate a unique one when necessary.

import type * as CSS from "csstype";

type CssLikeObject =
  | {
      [selector: string]: CSS.PropertiesHyphen | CssLikeObject;
    }
  | CSS.PropertiesHyphen;

We use the csstype library to set up the basic css types. CssLikeObject is either some css representation or some nested CSS representation object.

Here’s an example of what a CssLikeObject may look like:

{
    "font-size": "10px",
    ":hover": {
        "font-size": "20px"
    },
    ".randomStuff": {
        "font-size": "30px"
    },
}

As you can see, the nestable representation allows us to declare richer CSS rules that supports things like selectors and psudoelements.

Convert from JS representation to CSS string

But wait, how do we convert from CssLikeObject to string? First, let’s visualize how this CssLikeObject can be turned into css.

Assuming that the generated class name is .gen-class-1, the final output should be:

.gen-class-1 {
  font-size: 10px;
  color: green;
}
.gen-class-1:hover {
  font-size: 20px;
  color: red;
}
.gen-class-1 .randomStuff {
  font-size: 30px;
  color: blue;
}

We present an algorithm below to generate a set of CSS rules, each represented by a string. Comments inline.

function nestedDeclarationToRuleStrings(
  rootClassName: string,
  declaration: CssLikeObject
): string[] {
  const result: string[] = [];
  // We use a helper here to walk through the tree recursively
  function _helper(selector: string, declaration: CssLikeObject) {
    // We split the props into either nested css rules
    // or plain css props.
    const nestedNames: string[] = [];
    const cssProps: CSS.PropertiesHyphen = {};

    for (let prop in declaration) {
      const value = (<any>declaration)[prop];

      if (value instanceof Object) {
        nestedNames.push(prop);
      } else {
        (<any>cssProps)[prop] = value;
      }
    }

    const lines: string[] = [];
    // Collect all generated css rules.
    lines.push(`${selector} {`);
    for (let prop in cssProps) {
      // collect the top level css rules
      lines.push(`${prop}:${(<any>cssProps)[prop]};`);
    }
    lines.push("}");

    // Each string has to be a complete rule, not just a single
    // property
    result.push(lines.join("\n"));

    // Go through all nested css rules, generate string css rules
    // and collect them
    nestedNames.forEach((nestedSelector) =>
      _helper(
        joinSelectors(selector, nestedSelector),
        (<any>declaration)[nestedSelector]
      )
    );
  }
  _helper("." + rootClassName, declaration);
  return result;
}

We also introduced a joinSelector function because we need to treat states and psudoelements differently from descendant selectors.

function joinSelectors(parentSelector: string, nestedSelector: string) {
  if (nestedSelector[0] === ":") {
    return parentSelector + nestedSelector;
  } else {
    return `${parentSelector} ${nestedSelector}`;
  }
}

Generating class names

We haven’t really talked about generating class names yet. Here we are going to go with a simple counter. In some cases a stable hash may be better, but for our little toy example this is sufficient.

Having a counter introduces some states. Plus, we need to create a style element, which is itself a state. It’s time to make a class for all these. We are going to call this StyleDef.

class StyleDef {
  #selectorCounter = 0;
  #sheet: CSSStyleSheet;
  #prefix: string;

  constructor(prefix: string) {
    const sheetElement = document.createElement("style");
    document.head.appendChild(sheetElement);
    this.#sheet = sheetElement.sheet!;
    this.#prefix = prefix;
  }

  staticStyle(declaration: CssLikeObject): string {
    const selector = this.#generateSelector();
    const ruleStrings = nestedDeclarationToRuleStrings(selector, declaration);
    ruleStrings.forEach((ruleString) =>
      this.#sheet.insertRule(ruleString, this.#sheet.cssRules.length)
    );
    return selector;
  }

  #generateSelector = () => {
    return `${this.#prefix}-${this.#selectorCounter++}`;
  };
}

We also introduce a prefix string so that multiple StyleDefs won’t create classes that conflict with each other.

With that, we are done! You can play with the code in here .

Issues with CSS-in-JS in general

Now that we have built a small library, it’s time to talk about some issues with CSS-in-JS frameworks.

Performance

We can notice a few issues with our little library:

Instead of downloading and potentially parsing JS and CSS file in parallel, style code now needs to be parsed with JS code in series.
There’s some overhead in first generating CSS strings through JS, before having them parsed by the browser, compare to just giving the browser some raw CSS files.

The point is that there’s always some overhead in using CSS-in-JS compared to loading CSS files directly. For a small to medium project, this is likely not a problem, and the ease of development will likely outweigh the performance cost. For any massive projects, this can mean a large performance hit.

Build step to the rescue?

One solution here is to use CSS-in-JS during dev, but compiles them into CSS for build. However, this means that there is now a build step that needs maintaining and takes time. This build step is also potentially more complex than other css preprocessors like SCSS, because it needs to read JS/TS code, which has their own dependency trees and then emit CSS code, then replace some of the JS code output with the emitted CSS class name.

Additionally, using a build script usually means that dynamic styles has to be supported by CSS variables, which introduce its own limitation. The now deprecated emotion static extrator and linaria are two examples.

Ease of debugging

It used to be that CSSStyleSheetAPI takes away our ability to edit style rules directly in browser inspectors. This has since been fixed in Chrome and Firefox, though it’s still an issue in “the new IE” .

How about Inline styles?

This is one other approach in the CSS-in-JS field. However, it has two issues:

A lot of CSS features aren’t available - no pseduelements, selectors, media queries, states etc.
Every single element defines its own styles - this can lead to some concerns on performance due to a larger DOM representation per node and likely more DOM mutations

On the other hand, based on what we have just discussed, inline styles also has some perf advantage over our approach - we get to directly manipulate styles by modifying attributes of element.style rather than getting the browser to parse our CSS strings, though some parsing still has to be done as the browser needs to parse the property values.

Radium is an example that uses this approach and works around 1 by listening on DOM events to trigger special state or media queries.

ES6 tagged templates

A lot of the popular CSS-in-JS libraries allows for syntax like this

const Button = styled.a`
  color: blue;
  ${(props) =>
    props.primary &&
    css`
      color: green;
    `}
`;

This is done with ES6 tagged templates , which we will explore in a future post.

Things to look forward to

Why on earth can’t CSSStyleSheet support an API like inline styles, where we can set properties directly instead of having to pass raw css strings around? Well, CSS Typed Object Model does exactly that. However, it’s still in experimental stage, and only Chrome and its bastard child Edge support it as of now.

Easy and Quick Tip Calculator

Mon, 01 Jan 0001 00:00:00 +0000

A simple and modern tip calculator.

Privacy Policy for Easy and Quick Tip Calculator

Mon, 01 Jan 0001 00:00:00 +0000

Privacy Policy

This policy applies to all information collected or submitted by the Easy and Quick Tip Calculator application for iPhone

Information Collection

Easy and Quick Tip Calculator does not collect or retain any personal identifying information as part of its core functions.

Apple may collect crash logs if the privacy settings of your device allow it.

Children’s Privacy

Easy and Quick Tip Calculator is not designed to attract anyone under the age of 13. No identfying information is retained by Easy and Quick Tip Calculator regardless of age.

By using Easy and Quick Tip Calculator, you consent to this privacy policy.

Contacting Us

If you have questions regarding this privacy policy, you may email support@yyjhao.com .

Changes to This Privacy Policy

If this privacy policy changes, updates will be available on this page.

This policy is effective as of June 26, 2022.

yyjhao

Free and Secure Messaging on Raspberry Pi with Telegram Bots

Use Case

Why use telegram bots

Walkthrough

Setting up the bot

Quick start

Sending messages

Responding to messages

Complete code:

Set up rust + wasm + TypeScript without webpack

1 Set up the rust code and wasm-pack

2 Set up the TypeScript code

3 Set up a development iteration process

4 Set up a production build process

Set Up A Raspberry Pi As An Auto-refreshing Web Dashboard

Set up a Raspberry Pi to run Chromium in kiosk mode

Set up

Configuration

Create a service to (re)start Chromium

Roll Your Own CSS-in-JS Library (5) - Tagged Templates

CSS-in-JS but using CSS syntax

What is styled``

ES6 tagged template

CSS variables again?

High level approach

Defining the syntax

Visitor pattern

Parsing and visiting

Converting to flat CSS with CSS variables

Wrapping up

Build

Roll Your Own CSS-in-JS Library (4) - Static Extraction

Static extraction is HARD

Building a simple static extractor

Basic idea

Parsing JavaScript code

Identify valid styled calls

Statically evaluate each styled call

Generating the transformed JavaScript code

Complete source code

Last words

Roll Your Own CSS-in-JS Library (3) - Using CSS Variables

End result

CSS variables

Approach

Update to CssLikeObject

Rebuilding nestedDeclarationToRuleStrings

Issues

Does not support IE

Large generated stylesheet

Restricted syntax

Performance overhead?

Roll Your Own CSS-in-JS Library (2) - Dynamic Styles

End result

Dynamic class names

Issues

Unbounded number of class names

No more descendant selectors

Generating CSS code during build time is hard

Large generated stylesheet

Optimization - Atomic CSS

Implementation

React factory

Are dynamic styles really necessary?

Other ideas

Roll Your Own CSS-in-JS Library (1) - Introduction

The basic idea of run-time static CSS-in-JS

End Result

Using the CSSStyleSheet API

Example implementation

Defining our in-JS representation

Convert from JS representation to CSS string

Generating class names

Issues with CSS-in-JS in general

Performance

Build step to the rescue?

Ease of debugging

How about Inline styles?

ES6 tagged templates

Identify valid `styled` calls

Statically evaluate each `styled` call

Update to `CssLikeObject`

Rebuilding `nestedDeclarationToRuleStrings`