Tools for fast (and less furious) frontend development
Omar Rizwan
,
Engineer
ā¢
Wednesday, February 2, 2022
You can think of a Āé¶¹“å map as a space which has certain Āé¶¹“å elements (drawings, pins, notes, data) that live inside it.
You can think of a Āé¶¹“å map as a space which has certain Āé¶¹“å elements (drawings, pins, notes, data) that live inside it.
And there are a bunch of that you can do to a Āé¶¹“å element: move it around, rotate it, change some properties on it, and so on.
Maybe that sounds obvious, but I actually think that it makes Āé¶¹“å very unlike a lot of software that is forms and buttons and and . Āé¶¹“å has these objects that live inside it. (Plus, a map is at least a little bit than all those other kinds of software.)
Āé¶¹“å is more like an environment or a space, with its own basic building blocks that are governed by certain āā. You might compare it to [which has vector elements], or [blocks], or [cells], or a text editor [characters], or to your filesystem or [files and folders], or even to a Web page [the that comprise the page].
I wanted to talk about some techniques for quickly developing and iterating on this sort of (object-y, visual, direct-manipulation) software. The following is a grab bag of general techniques which have paid dividends here at Āé¶¹“å.
1. Hot reload!
As I think many people in software now realize, iteration speed is all-important. You want to have immediate feedback into what your changes are doing to the software ā itās great for motivation, it lets you play and try new things, and come up with solutions on the fly that you wouldnāt otherwise think of.
Hot reload is interesting in that itās as much an aspiration as it is a particular technologyāyou need different technology to make different parts of your application āhot-reloadableā ā so you can spend an almost unbounded amount of time improving reload coverage (edits to React component source? changes to CSS? changes to map rendering code?) and improving reload speed (a minute? 10 seconds? 500 ms? 50 ms?).
At the moment, weāre at a middle point on that spectrumāwe have hot reload for React components and for our Elixir backend, but some CSS top-level theme changes require a refresh, for example. Weāll probably improve on that as our styling gets more complicated and we want to tweak it more continuously.
Thereās a human element to how you should set up hot reload. Is someone in the organization making a lot of a certain kind of change? Are they blocked on waiting for the thing to reload, or / could they design better with live feedback? Maybe you should fix hot reload for that area; unblocking them will probably produce a lot of value.
Itās worth noting that hot reload shouldnāt be magic or āset and forgetā; as a developer, you generally want to know how the hot reload system works, because it will fail or go wrong at some point. You need to know what classes of things you can do āhotā and what classes of things will glitch out or require a restart. A lot of that is to the context of your project and . In our experience, the fast feedback loop is worth this price.
Want to know more about Āé¶¹“å?
Learn all you need to know watching this 2 minute video or book a demo with our team.
We write all our frontend code in TypeScript, but the hot compilation doesnāt actually use the TypeScript compiler ā it runs the code through and doesnāt do typechecking. That means that, first of all, itās much, much faster than using <p-inline>tsc<p-inline>.Ā We can hot reload in a few hundred milliseconds at most (and I think we could go faster if we pushed). We also have git pre-push checks and continuous integration and editor support. This keeps us aware of type errors, allows us to check the error log as needed, but the errors don't interrupt the critical path of compilation and development work. They appear asynchronously, on the side, from the TypeScript language server, and at push time.
Equally as important, it doesnāt feel like typechecking is a blocker for development; you donāt need to care about the type errors if youāre working on something; so you can do classic dynamically-typed programming/exploration/probing /playing around early on:
I want to see whatās in this object (or if it changes over time!); Iāll assign it to <p-inline>window.foo<p-inline>, and poke at it in the browser console at runtime.
I want a bit of context/provenance when something happens later in the program, so early on, Iāll tag some data on as an extra property <p-inline>._extraStuff<p-inline> onto this object thatās getting passed through my program already.
I just want to see if this thing will work at all, so Iāll ignore the null case or error case for now (or I wonāt bother to give the right type, or whatever)
You can fix up the types when youāre ready to commit/pushāor when you run into a roadblock and youāre like, āokay, maybe Iāve missed something,ā and you consult the type error log. The type errors are useful; donāt get me wrong! They just shouldnāt necessarily hold you up all the time or distort the way that you program.
This technique comes out of a property of TypeScript that Iāve always found interesting. Historically, the point of a type system was to help the compiler to generate code (e.g. is this variable 8 bits or 32 bits in memory?). TypeScript, in contrast, has nothing to do with how a JavaScript VM executes your code; it erases all the type information and just emits plain JavaScript in the end. TypeScript is basically a sort of amped-up linter that is there to help you, not to help the runtime.
3. Development Server Over the Internet
Weāve started using , which is a service that lets me take the https://localhost:4000 development server on my laptop and put it right up on the Internet at a URL that looks like . If you want to pair program on something with someone, you can just send them that ngrok URL, and theyāll be connected, live, to the server on your computer.
Hot reload works here in the exact same way that it does on your local computer. This is really powerfulāyou edit the source file on your computer, and it immediately hot reloads in your browser and in the browser of whoever youāre pairing withānow you tweak a shade of blue or an animation, and they can play with it on their end, in their own way (maybe they are screen-sharing their screen so you can see what they see). The traditional approach of sharing your screen is much inferior to this, I think; it doesnāt have that ability of independent interaction; it drains the other person of agency. They canāt look around and notice other things that are broken.
I also think that the over-the-Internet hot reload creates this feeling of connection, that youāre both touching this live wire, and thereās something really compelling about that. Not only does screen sharing dilute or prevent interaction, it doesnāt seem to create this same feeling.
There are also the advantages of code consistency and data consistency: if weāre pairing about some problem, and weāre both using my development server, and we refresh the page, we can be pretty sure that 1. weāre looking at the exact same map data and 2. weāre using the exact same version of the application. Weāll talk later about import and export as another route to data consistency.
4. 'Observability'
I talked about the desire for immediate feedback earlier. Immediate feedback doesnāt just mean that the app reloads really fast. Immediate feedback means you should be able to see the thing that you care about immediately. The thing that you care about is not necessarily just āthe app interfaceā or āwhether it worksā; in fact, while youāre programming, you often want to see , or you want to see how some performance metric is changing, etc.
Frontend programmers probably donāt do enough of this yet. React developer tools provide some of this observability, but itās too clumsy, in my opinion (youāre clicking around this giant React component tree and squinting at blobs of JSON with latitudes and longitudes in them, instead of seeing the graphical Āé¶¹“å elements and the places on the map).
Ideally, weād have an inspector thatās as powerful as the Web inspector, but it (instead of DOM elements, Āé¶¹“å elements; instead of a component tree, an actual geographical map?).
One concrete thing that we did toward this Āé¶¹“å-inspector dream: we exposed some variables, felt.$ and felt.$0, in the browser console.
You might know about $ and $0 in the browser console: these are variables that give you programmatic access to the DOM element that you've selected in the Web inspector.
It's really nice to be able to point at something in the DOM, then program with it (ask questions about its properties, mutate it, etc). I wanted something similar for our Āé¶¹“å elements ā you should be able to point at a Āé¶¹“å element and then have interactive access to it from the JavaScript console.
It's been really useful to debug changes we make to the drawing system in Āé¶¹“å. These are the kinds of tools that we needātools that provide us the ability to inspect an element, see its raw data (and if that matches up with what our frontend is showing), perturb it in various ways.
It also hints at the power of the browser console as a tool for developing applications, which I want to talk a little more aboutā
5. Browser Console
I think that a tragedy of modern, , rich Web applications is that they destroy a lot of the utility of the browser JavaScript console and the Web inspector. These are really powerful tools!
Your CSS styles all live in classes with obfuscated names (generated by some CSS-in-JS system), the CSS inspector isnāt designed to deal with that / even if you can tweak them in the inspector, youāve lost most of the structure that you wrote in the original code
Your applicationsā data is in you have to set a breakpoint to even get to a point where you can see what your application is doing
Transpiled variable names are mangled and you canāt even read them unless you also turn off source maps
Stack traces donāt work properly over async-await, or across a chain of React calls, or whatever, so youāre using console.log and breakpoints to reverse-engineer control flowĀ
Anyway, the browser console is an excellent way to get more visibility into what the application is doing, and to do lightweight programming and scripting and analysis (without needing to edit the app itself).
ā Itās a nice JS environment which already has a lot of the context (all the normal libraries that we use in Āé¶¹“å can be imported, thereās a map open in the browser that you can take context from and draw onto, etc). Much easier than trying something in node at the command line, and much easier than actually developing inside the actual app and having to wire it up with the UI and with React and everything.
Also ā as you can see above, the browser console supports CSS styles! It isnāt only the plain text that you might expect.
Something you might not know is that you can actually You can render into an HTML5 canvas, convert that canvas to a data URI, and then set that data URI as a CSSĀ <p-inline>background-image<p-inline> in <p-inline>console.log<p-inline>. This is really useful for us (and, really, for any application where there is a visual-spatial element and you want to actually see what youāre developing).ā
need to get more into this
— Omar š Vancouver, BC (@rsnous)
Sometimes Iāve developed things purely as imperative browser commands first, and I get the logic right in that self-contained context (where I can scroll up and compare to past results!), and then I add it to where it belongs in the Āé¶¹“å app.
For example, I pan and zoom the map to the place where I want to test something, maybe draw some stuff in the app, then open up the console and iterate on some command that uses that location and/or the elements on the map.
The browser console also provides a nice interface to expose debug functionality without you having to make a proper HTML UI. You can make global functions to turn on debugging modes, to reveal different pieces of data, and so on, which leads us toā
6. Import and Export Data
Āé¶¹“å has a pretty complicated document model! Maps can be in all kinds of different states with all kinds of different objects with different properties/scale/location/zoom.
If I run into a problem, or I have some half-developed feature, and I want you to check it out, I want to be able to send you a copy of the map that Iām talking about, so you can look at it on your computer and try different things.
One solution is what I discussed earlier: be on the same server with the same backend database (which means either the production server, a development server deployed off a GitHub branch, or someoneās local development server over ngrok).
When thatās impractical, though, itās also very useful to be able to quickly import and export objects; you can paste them into Slack or a GitHub issue comment or whatever you use (itās also useful for debugging to be able to see what the ground-truth data looks like).
So we added browser console commands that do this (it only took a few lines of code, purely on the frontend): we can now dump Āé¶¹“å objects as a chunk of JavaScript, and we can paste that chunk into any other Āé¶¹“å mapās browser console to import those objects.
You want to have immediate feedback into what your changes are doing to the software ā itās great for motivation, it lets you play, try new things, and come up with solutions on the fly that you wouldnāt otherwise think of.
7. Build the Frontend the Backend in Parallel
Build all your views and interactions and transitions for real, but have them backed by state that lives purely on the client at first (in React local state, or in whatever state manager that you use ā we use jotai).Then, later, when you have your backend working, switch to using your backend as the canonical state instead.
Sometimes it really is that simple! It can be, like, a five-line change. (Sometimes it takes a bit more work to migrate to using server-side state: weāve had cases where the server wants to asynchronously change the data over time for security/resource use/postprocessing reasons, for example. We ran into this with image uploading: the client should render the uploaded image immediately, so it feels fast, but the server may replace it later as it postprocesses the image.)
There is some risk to this ā that your frontend wonāt quite be aligned with your backend ā but it has worked well for us. I think you need the right framework and frontend data model to allow you to cleanly swap between frontend and backend state. I think that there is a huge psychological advantage to working this way, where you never feel blocked on anything on the frontend, and it doesnāt feel like a as youāre working on the interface because you have to change a bunch of points in the backend at the same time.
8. Start with the Prototype
This applies mostly if you have a design prototype from something like Figma or Sketch (or even a hand-drawn prototype). When you start developing that view for real, take that prototype, turn it into an image (a PNG file, etc.), and put that image right in the app (probably as a CSS <p-inline>background-image<p-inline> for the view that youāre working on). (You might set the CSS <p-inline>opacity<p-inline> for your real view to <p-inline>0.5 or something<p-inline>, so you can see through it to the prototype underneath and check how well they align.)
Then your development process is a process of continuously āfixingā your real view, tuning it to match the prototype underneath, rater than trying to make it from scratch. Youāre taking more and more of that dead static mock and tracing bits out and bringing them to life. A problem with this is that you need to have a good handle on your abstractions and primitives to do it well. Itās easy to just spew out hard-coded pixel values that make it match, but then youāre obscuring the structure of the design and making it brittle and hard to understand and change. You also donāt know how itāll respond to resizes and theme changes and animation and so on.
Constant and Fast Iteration at Āé¶¹“å
At Āé¶¹“å, we believe the best products are built with constant and fast iteration. We go the extra mile at every point, whether it is cartography, frontend, infrastructure, to lower the distance between having an idea and seeing it live on screen. As a company that builds mapping products on the web, the frontend iteration cycles are especially important. In this post, I tried to list some of the less obvious ways to squeeze out some feedback loops. This work is exciting, but we are nowhere near done! We are always looking for amazing people to .
I think that there is a huge psychological advantage to working this way, where you never feel blocked on anything on the frontend, and it doesnāt feel like a chore as youāre working on the interface because you have to change a bunch of points in the backend at the same time.