Whole blog post is basically: Make a mutation in the clientside, assume it worked, and save in the background.
mohsen1 29 minutes ago [-]
Works for Linear because the tab stays open, and worse case if tab is closed you can recover later when the tab is opened again and deal with conflict resolution. Won't work if:
1. user clicks a button and closes the tab thinking transaction is done and it's important that transaction is done
2. conflict resolution is difficult or impossible in future client wake up
nine_k 18 minutes ago [-]
The user clicks the button, the mutation is stored in local storage. The user closes the tab, but it's not a problem.
A background worker picks up the mutation and sends it to the remote backend. It takes time, retries, etc.
Similarly, any errors reported by the background worker go to local store, and the next time the UI tab is loaded / activated, they are shown. A service worker can show a notification outright to let the user easily load the main UI. Normally this would be a rare occasion.
ifwinterco 2 minutes ago [-]
Yeah this pattern can be made to work fine.
Main downside is it significantly complicates the front end code compared to just waiting for FE to sync with BE before updating
Onavo 7 minutes ago [-]
Yes and for linear (if you break it down strictly in a theoretical CS sense, is roughly equivalent to TodoMVC in terms of application complexity assuming non-collaborative text editing) they have clearly defined states for most items and few if none truly destructive actions. The hardest part is anything text related.
NewsaHackO 56 minutes ago [-]
I agree. To each their own, but the UI updating automatically doesn't really add much value to me. I would prefer that the view I am seeing is a snapshot in time of what the ground truth server was, not some mixed state that forces me to consider the possibility that seeing my request go through on the screen doesn't actually mean it went through and has been sent to the server.
sneak 6 minutes ago [-]
I believe this is called “eventual consistency”.
pier25 53 minutes ago [-]
For native apps this is less of an issue since they have access to persistent storage but with browsers there's no guaranteed persistence.
WhyNotHugo 30 minutes ago [-]
There's guaranteed persistence, but there's no guarantee that the host will be up anytime soon. E.g.: I might leave a final reply with all the details on an issue before going on vacation (or maybe I don't work the next day but my colleagues abroad do!). I see that it's properly posted and close the laptop.
The reply with be delayed by days or weeks, but the UI indicated that it had been properly saved.
pier25 27 minutes ago [-]
> There's guaranteed persistence
There's not. Browsers can delete "persistent" storage at any time.
> If, for any reason, developers need persistent storage [...] they can do so by using the navigator.storage.persist() method of the Storage API.
This makes a request for guaranteed permanent storage ... which can be approved (or denied) by the user or by browser defaults.
prerok 55 minutes ago [-]
Indeed. I have to say, I hate this. Suppose you are in a meeting, you update something and you see the result, but the rest of the team does not. Ok, a couple of hundred ms does not play into this but if the update does not make it through? And yes, it happens.
artman 28 minutes ago [-]
Changes go through and synced to everyone on your team in almost realtime. If there's a conflict on the server and your change cannot be applied (almost never happens), your change is rolled back on your client, again, almost in realtime. If servers cannot be reached, we will show you a syncing badge within 4 seconds to tell you that you have made changes that haven't been sent to others yet.
Strange that we can be so be polar opposites on this. You hate it, I would never write an app in any other way, ever again.
n_e 4 minutes ago [-]
As a user, I like when things appear to sync instantly and perfectly, such as in Google Docs.
As a developer, I hated the article and many of the comments I read thus far because:
- Having clients and a server properly sync and not lose data in the event of a network failure amounts to having a consistent distributed system which is not easy to do, and the commenters don't seem to have understood that
- I hate having written a long document and then losing it because the sync code is buggy, so the previous point becomes even more important.
So reading many of the things here has been mildly infuriating.
That being said, none of these people are likely affiliated with Linear, and given the overall quality of the product I'm pretty sure it works properly.
porridgeraisin 22 minutes ago [-]
(curious) What if a user closes it before 4 seconds? Ctrl+enter, it optimistically locally updates within 1 second. I close ctrl+w. But my wifi goofed and it didn't reach the server.
jacobgold 1 hours ago [-]
These kinds of local-first syncing web apps are really interesting and can be really useful, but I think the premise is somewhat wrong.
"A few milliseconds is all it takes to update an issue in Linear. A traditional CRUD app doing the same thing takes about 300ms."
"Any data sent between the client and server costs hundreds of milliseconds."
There’s no solving the problem of a large RTT between an HTTP client and server when it’s due to the speed of light.
But what you can do is locate the backend near users and make sure it’s fast.
For example, it’s very possible to run a web app backend within ~10ms RTT of most users and have the backend render responses within ~10ms too.
In other words, you can absolutely create a traditional CRUD app where doing the same thing takes more like 30ms, not 300ms.
nine_k 22 minutes ago [-]
You can locate an "intermediary backend" on the client, write outstanding mutations into local storage, have a background worker send it to the backend, with necessary retries, etc. At worst, the background worker would put out a message about a failed update which the UI tread would receive and show.
But the happy path stays lightning-fast.
jacobgold 10 minutes ago [-]
Sure but there's a ton of complexity in any kind of local-first syncing solution. Often the solution is CRDTs.
My point above is that the simple solution ("traditional CRUD app") is actually viable even when the goal is very low latency.
44 minutes ago [-]
Escapado 50 minutes ago [-]
I love using linear and where applicable I try to use optimistic updates in the applications I build too and I think it's a good article that touches on the most important aspects of a performant web app of that caliber.
One thing I would like to point out though is that building a performant sync engine that behaves the way you would like in most cases is a non-trivial thing.
If two users are offline and add, edit, remove issues and come online again, you need to reason about what happens. Sometimes you can get away with Last Writer Wins but what happens if an issue is deleted and then edited. What happens if an item in a list gets re-ordered differently on different clients, what's the final ordering? In which cases can you merge what state and in which do you need to discard something. Do you show a conflict resolution UI? How do you deal with rollbacks. How do you deal with schema drift and updates on items that would be affected by schema drift? Business logic might change between being reconnects. FK constraints can shift. Can you set up your data and the sync engine so that it only syncs the minimum amount of changes and batches them correctly during longer offline sessions so you don't fire 5000 change requests after reconnecting?
I recently had to implement local first + remote sync on some fairly complex dynamic forms and where luckily there is usually only one writer and I can get away with last writer wins and reject if things are too old or if there is schema drift and can just display an error message and roll back. But what I am trying to say is: Whatever can go wrong in an online-first world, can also go wrong in an offline first world but you might get informed of that all at once at a later time - or not at all and your data is not what you think it should be. Some sync engines like zero from rocicorp has opted out of supporting offline writes entirely because of all these problems.
And just to be clear: I love offline first approaches. I yearn for fast performance. And in a lot of cases slapping a sync engine on your app can really be helpful for that if your use case allows it. But it's absolutely crucial to be aware of the pitfalls that come with it.
cuechan 1 hours ago [-]
> A traditional CRUD app doing the same thing takes about 300ms.
300ms seems like a lot. Even if this includes the whole http request+response it still seems unbelievably long.
jpollock 56 minutes ago [-]
It depends on where your clients are, and where your servers are. You may have a lot of customers in a country and not want to host servers in that country (tax, regulatory, maintenance cost, etc.)
RTT from Hyderabad to the East Coast USA is ~300ms.
this is kinda missing the point. yes, the raw http calls __should__ be enough for 300ms.
but the benchmark is similar software like JIRA, which takes agonizingly long to do anything reasonable.
IshKebab 36 minutes ago [-]
Yeah you can easily get this speed down to like 100ms which is near enough to instant to not matter. You definitely don't need to go to client side rendering for most things to feel fast.
Also I couldn't see any explanation of what happens when a network request fails. That's a huge downside of local rendering.
I think for most sites the best option is still server-side rendering but use a fast backend (e.g. not Python) and lightweight frontend.
wbobeirne 55 minutes ago [-]
I'm glad to see more positive takes on client-side rendering. Unless SEO and crawlability are important for your site, server side rendering is such an overprescribed solution. If I put my tin-foil hat on, I'd say it gets a lot of attention because it's a lot easier to charge people for server time spent in SSR that you just don't have in CSR.
Bottom line is, if your app's content is behind a login screen, just use client side rendering. It is way lower complexity and a way better user experience.
hamandcheese 56 minutes ago [-]
I worry that optimistic updates is going to become trendy and applied to more software, but without any plan for the "sad path" - failed to sync, sync conflict, etc. Get ready for a whole new era of race conditions and frustration!
MaoSYJ 7 minutes ago [-]
well it has been the standard for non critical feedback.
I would not implement optimistic resolution in key information, prices, for example. They live in the server and backend should keep the total control in the client side, even feedback response time.
12_throw_away 15 minutes ago [-]
> optimistic updates [...] without any plan for the "sad path"
based on my experience, this is a great description of the sync implementation in many already widely deployed products (say, off the top of my head, OneNote, OneDrive)
jack_pp 51 minutes ago [-]
Don't you have the same problems with basic CRUD apps? Also you need to handle the sad path for every single request instead of having the sync engine do it all in one place.
Escapado 47 minutes ago [-]
Correct but the feedback is usually more immediate. Save a change to your issue and it fails - You will get an error toast and probably stay on the form.
In the local first world you might have navigated away already and created 3 more issues of which 2 more failed because of schema drift or other conflicts. And you might have edited one that was deleted. And now you need to figure out what exactly to tell the user - or what not to tell them.
jack_pp 38 minutes ago [-]
Well the sync engine can figure out if there's an issue fast, say <500ms, if you build it that way. Then you can just make a toast telling the user there are issues and anything they do will be saved locally only for the time being.
Warn the user that if they leave the website their changes won't be saved remotely.
artman 27 minutes ago [-]
In reality conflicts almost never happen.
hamandcheese 34 minutes ago [-]
The issue that I foresee is that the point of error becomes decoupled from the UI and the UI doesn't handle a delayed error. Especially if retrofit into existing products.
epgui 26 minutes ago [-]
I’ve never thought of Linear as particularly fast.
esafak 25 minutes ago [-]
What web app would you put forward instead?
ianberdin 1 hours ago [-]
Thanks for the read. It is a bit more complicated than you think.
I completely rebuilt this sync engine + orm with relations, lazy loading etc, using Vue + Pinia in https://playcode.io. Google Linear’s videos, they explained in detail their architecture.
Yes, I spent a few months. But it worth it. Every new field, model I need to add, it is so straightforward. I do love frameworks and foundations. They make live easier later by a lot.
ianberdin 1 hours ago [-]
Feel free to ask questions.
lampe3 1 hours ago [-]
what google linear video?
echoangle 1 hours ago [-]
I think google is an imperative verb here. You're supposed to google (as in search for) Linears Videos.
These are original video. Very clear and a way better than the blog post.
bfung 3 minutes ago [-]
[delayed]
1 hours ago [-]
nilirl 17 minutes ago [-]
Am I the only that doesn't "get" Linear? Even if speed is the killer feature, isn't an issue tracker a relatively low-frequency application for a dev?
Whenever I use it, I don't feel like I'm doing anything new when compared to all the other issue trackers and Kanban boards I've used before.
andersonklando 7 minutes ago [-]
It is definitely way more performant than Jira + Microsoft Planner. Trello comes to mind in terms of performance. I speak from being a heavy user of Linear for my personal project and as a Jira enterprise user.
A leader of 6 will spend a lot of time in such an app, so the UX is valuable and a differentiator for them.
ambicapter 12 minutes ago [-]
I honesty can’t say it’s better than Jira, the myriad options just make it a confusing mess to figure out how to navigate and put stuff (could be my company is just hella disorganized), and the GitHub tracking is annoyingly eager (just because the first PR has been merged doesn’t mean the ticket is done).
yolo3000 3 minutes ago [-]
Maybe we work at the same company. I have the same experience. Don't know why people use it.
adverbly 1 hours ago [-]
Any good resources or wrappers for indexedDB people would recommend? The API seems kinda unapproachable compared to other data stores.
In gamedev, "optimistic updates" are called "client-side prediction," and are a standard part of multiplayer games. IMO it's somewhat risky to apply the technique to web-apps, since each network request typically corresponds to some important operation, and optimistically updating the UI is lying to the user about whether that operation completed successfully.
IMO a good approach is to update the UI immediately but still show some indication that the operation hasn't completed. So in a chat app, for example, add the message to the list of messages, but with contrast reduced slightly to indicate that other people can't see it yet.
artman 24 minutes ago [-]
Operations are on average applied within a few hundred milliseconds, and almost never fail. Because of this we treat the success path as default, and indicate that your changes haven't been applied only if we detect that you're offline, or if it takes more than 4 second to apply the changes.
Honestly you can POST to an API to the other side of the world and receive a response in less than 300ms. That's literally the blink of an eye. There are very few use cases where you'd need even lower latency than that while at the same time sacrificing reliability (there's no guaranteed persistent storage in a browser).
saagarjha 57 minutes ago [-]
I’ve actually found Linear to be quite slow? There was a week where it would also spin at 100% CPU if I left a tab with it open for a while :(
seabombs 51 minutes ago [-]
It appears to use a lot of memory too, at least in Firefox. I can't have more than a few Linear tabs open at a time.
jeffbee 24 minutes ago [-]
This is bizarrely laudatory. The app is fast because it is not correct. The user has no way to know if their view is consistent with any other user's view. The user has no way to know if the app silently discarded one of their inputs because of a conflict. Linear developers seem to believe that silent data loss is an acceptable cost for maintaining the illusion of speed.
blauditore 49 minutes ago [-]
TL;DR: Background syncing instead of synchronous updates to the cloud.
This is basically a thick client, and comes with according trade-offs. It's interesting and there are some best practices, but I can't help but feeling that either the author is a huge fan or the post is an ad (or "sponsored").
lampe3 1 hours ago [-]
funny how things come back.
I loved meteorjs and its still my fav framework to this day.
Seeing optimistic ui and browser based databases being used this way makes me happy.
I would like to understand how they do the chunking and only updating part of the graph instead all of it.
AG342 1 hours ago [-]
I've always wondered how they made the UX so slick - thanks for sharing.
ralferoo 33 minutes ago [-]
I've never used linear, but just watching their example video [1] would worry me if I was a user and it's actually doing what they say it's doing.
He creates a task called "Create faster app launch", if we believe the article, it's processing that locally rather than going via the server, and then it's allocated an ID "BRO-5". That the ID is so low suggests it's just adding one to the previous issue ID, and so under heavy load, there are almost certainly going to be conflicts with other users creating tasks and getting identical IDs. Even if the system resolves this by changing one of those IDs, the system shouldn't be presenting the ID to the user until it is guaranteed unique. What if they've already pasted it into a document when the system notices the collision and renumbers it?
EDIT: WTH, there are some seriously bad karma people in this thread - just because I dared to have an opinion that the approach taken by this software might not be the best, my post was downvoted in less than a minute after posting! I'm sure whoever did that carefully considered my argument. If I'm wrong, explain why, don't just downvote my comment. If I'm not wrong, shame on you.
ArtRichards 1 hours ago [-]
They're doing some really cool stuff with Agents too.. automating software pm. it's a huge difference between this and their (unnamed) larger competitors.
xiaoyu2006 1 hours ago [-]
I was expecting a post on how hardware accelerate linear algebra lol
0xbadcafebee 47 minutes ago [-]
> Literally the first lines of code that I wrote was the sync engine, which is very uncommon to what you usually do when you're a startup
He chose the path to a better product rather than the path to a quick buck. That is definitely odd for a silicon valley startup
1 hours ago [-]
sieabahlpark 1 hours ago [-]
[dead]
jwr 39 minutes ago [-]
Linear is the best web app I have ever seen, period. It is also the best bug-tracker I've ever used. I use it, and pay for it (gladly). It's worth it.
I am genuinely impressed with their engineering and design — I aspire to attain these levels, though I lack not only the skills, but also several zeroes in my bank account, I think. Still, it's worth looking at what they do and try to get there!
Big props and kudos to the Linear team. It's an impressive app.
apsurd 25 minutes ago [-]
Interestingly, as a professional developer I had the opposite first experience. It's a tracker. It looks like every other tracker. Everything is really small and "clean" which means they just hide everything, for that modern look.
I was thoroughly unimpressed. "This is it?" - "It's a tracker"
I feel comfortable saying this because in the weeks after, actually using it is the aha experience. Linear nailed the UX of working where people already work. Which again is really funny because the best part of Linear is how well it works outside of Linear.
(disclaimer: I actually now use their UI a lot. It's a helpful dashboard. But it suffers from every other hard-problem of information dense task-based dashboards.)
https://github.com/wzhudev/reverse-linear-sync-engine/blob/m...
1. user clicks a button and closes the tab thinking transaction is done and it's important that transaction is done
2. conflict resolution is difficult or impossible in future client wake up
A background worker picks up the mutation and sends it to the remote backend. It takes time, retries, etc.
Similarly, any errors reported by the background worker go to local store, and the next time the UI tab is loaded / activated, they are shown. A service worker can show a notification outright to let the user easily load the main UI. Normally this would be a rare occasion.
Main downside is it significantly complicates the front end code compared to just waiting for FE to sync with BE before updating
The reply with be delayed by days or weeks, but the UI indicated that it had been properly saved.
There's not. Browsers can delete "persistent" storage at any time.
https://developer.mozilla.org/en-US/docs/Web/API/Storage_API...
> If, for any reason, developers need persistent storage [...] they can do so by using the navigator.storage.persist() method of the Storage API.
This makes a request for guaranteed permanent storage ... which can be approved (or denied) by the user or by browser defaults.
Strange that we can be so be polar opposites on this. You hate it, I would never write an app in any other way, ever again.
As a developer, I hated the article and many of the comments I read thus far because:
- Having clients and a server properly sync and not lose data in the event of a network failure amounts to having a consistent distributed system which is not easy to do, and the commenters don't seem to have understood that
- I hate having written a long document and then losing it because the sync code is buggy, so the previous point becomes even more important.
So reading many of the things here has been mildly infuriating.
That being said, none of these people are likely affiliated with Linear, and given the overall quality of the product I'm pretty sure it works properly.
"A few milliseconds is all it takes to update an issue in Linear. A traditional CRUD app doing the same thing takes about 300ms."
"Any data sent between the client and server costs hundreds of milliseconds."
There’s no solving the problem of a large RTT between an HTTP client and server when it’s due to the speed of light.
But what you can do is locate the backend near users and make sure it’s fast.
For example, it’s very possible to run a web app backend within ~10ms RTT of most users and have the backend render responses within ~10ms too.
In other words, you can absolutely create a traditional CRUD app where doing the same thing takes more like 30ms, not 300ms.
But the happy path stays lightning-fast.
My point above is that the simple solution ("traditional CRUD app") is actually viable even when the goal is very low latency.
One thing I would like to point out though is that building a performant sync engine that behaves the way you would like in most cases is a non-trivial thing.
If two users are offline and add, edit, remove issues and come online again, you need to reason about what happens. Sometimes you can get away with Last Writer Wins but what happens if an issue is deleted and then edited. What happens if an item in a list gets re-ordered differently on different clients, what's the final ordering? In which cases can you merge what state and in which do you need to discard something. Do you show a conflict resolution UI? How do you deal with rollbacks. How do you deal with schema drift and updates on items that would be affected by schema drift? Business logic might change between being reconnects. FK constraints can shift. Can you set up your data and the sync engine so that it only syncs the minimum amount of changes and batches them correctly during longer offline sessions so you don't fire 5000 change requests after reconnecting?
I recently had to implement local first + remote sync on some fairly complex dynamic forms and where luckily there is usually only one writer and I can get away with last writer wins and reject if things are too old or if there is schema drift and can just display an error message and roll back. But what I am trying to say is: Whatever can go wrong in an online-first world, can also go wrong in an offline first world but you might get informed of that all at once at a later time - or not at all and your data is not what you think it should be. Some sync engines like zero from rocicorp has opted out of supporting offline writes entirely because of all these problems.
And just to be clear: I love offline first approaches. I yearn for fast performance. And in a lot of cases slapping a sync engine on your app can really be helpful for that if your use case allows it. But it's absolutely crucial to be aware of the pitfalls that come with it.
300ms seems like a lot. Even if this includes the whole http request+response it still seems unbelievably long.
RTT from Hyderabad to the East Coast USA is ~300ms.
Then you have execution and database retrieval.
https://wondernetwork.com/pings
but the benchmark is similar software like JIRA, which takes agonizingly long to do anything reasonable.
Also I couldn't see any explanation of what happens when a network request fails. That's a huge downside of local rendering.
I think for most sites the best option is still server-side rendering but use a fast backend (e.g. not Python) and lightweight frontend.
Bottom line is, if your app's content is behind a login screen, just use client side rendering. It is way lower complexity and a way better user experience.
I would not implement optimistic resolution in key information, prices, for example. They live in the server and backend should keep the total control in the client side, even feedback response time.
based on my experience, this is a great description of the sync implementation in many already widely deployed products (say, off the top of my head, OneNote, OneDrive)
In the local first world you might have navigated away already and created 3 more issues of which 2 more failed because of schema drift or other conflicts. And you might have edited one that was deleted. And now you need to figure out what exactly to tell the user - or what not to tell them.
Warn the user that if they leave the website their changes won't be saved remotely.
Yes, I spent a few months. But it worth it. Every new field, model I need to add, it is so straightforward. I do love frameworks and foundations. They make live easier later by a lot.
https://m.youtube.com/watch?v=WxK11RsLqp4&t=2175s&pp=2AH_EJA...
These are original video. Very clear and a way better than the blog post.
Whenever I use it, I don't feel like I'm doing anything new when compared to all the other issue trackers and Kanban boards I've used before.
A leader of 6 will spend a lot of time in such an app, so the UX is valuable and a differentiator for them.
https://github.com/tinyplex/tinybase seems kinda good maybe?
IMO a good approach is to update the UI immediately but still show some indication that the operation hasn't completed. So in a chat app, for example, add the message to the list of messages, but with contrast reduced slightly to indicate that other people can't see it yet.
This is basically a thick client, and comes with according trade-offs. It's interesting and there are some best practices, but I can't help but feeling that either the author is a huge fan or the post is an ad (or "sponsored").
I loved meteorjs and its still my fav framework to this day.
Seeing optimistic ui and browser based databases being used this way makes me happy.
I would like to understand how they do the chunking and only updating part of the graph instead all of it.
He creates a task called "Create faster app launch", if we believe the article, it's processing that locally rather than going via the server, and then it's allocated an ID "BRO-5". That the ID is so low suggests it's just adding one to the previous issue ID, and so under heavy load, there are almost certainly going to be conflicts with other users creating tasks and getting identical IDs. Even if the system resolves this by changing one of those IDs, the system shouldn't be presenting the ID to the user until it is guaranteed unique. What if they've already pasted it into a document when the system notices the collision and renumbers it?
[1] https://media.performance.dev/posts/p_gAMR6Z7y49Fp/NZrXs70M_...
EDIT: WTH, there are some seriously bad karma people in this thread - just because I dared to have an opinion that the approach taken by this software might not be the best, my post was downvoted in less than a minute after posting! I'm sure whoever did that carefully considered my argument. If I'm wrong, explain why, don't just downvote my comment. If I'm not wrong, shame on you.
He chose the path to a better product rather than the path to a quick buck. That is definitely odd for a silicon valley startup
I am genuinely impressed with their engineering and design — I aspire to attain these levels, though I lack not only the skills, but also several zeroes in my bank account, I think. Still, it's worth looking at what they do and try to get there!
Big props and kudos to the Linear team. It's an impressive app.
I was thoroughly unimpressed. "This is it?" - "It's a tracker"
I feel comfortable saying this because in the weeks after, actually using it is the aha experience. Linear nailed the UX of working where people already work. Which again is really funny because the best part of Linear is how well it works outside of Linear.
(disclaimer: I actually now use their UI a lot. It's a helpful dashboard. But it suffers from every other hard-problem of information dense task-based dashboards.)