Not so long ago I was involved in the development of some hardware-software complex for an American company.I was working on the backend, a little bit of the frontend, I was splicing the devices to the cloud ( IoT that is). The technology stack was clearly marked. Neither right nor left – enterprise, in a word. At a certain point, I was thrown into helping with the frontend POS (Point of Sale) web application.
Issue. It gets more interesting
That would be OK, but the web app was designed to work in 6, 000 offices across America (for starters). Where, as it turns out, the Internet can be a problem. Yes yes, in that very, very advanced America! Problems with not only wired Internet coverage, but mobile coverage as well! That is, bad Internet (often, mobile) is quite a common story in small American cities.
And this is POS… You know, customers are standing here, they need to print invoices fast… There shouldn’t be any lags! There were a lot of discussions, some estimates, in the end – we didn’t want to load the backend with requests (traffic, again). Agreed that web app should load data as much as possible and do the same search locally. We are talking, of course, about data the size of which allows to do this.
Frontend was pulling a lot of data from different services. As a consequence – a lot of traffic and long loading pages. All in all, it’s a disaster.
Some problems are solved by backend (compression, geo-clustering, etc), but that’s a separate story, now we are just talking about frontend.
The first thing that was immediately done (and logically so) was to cache and store the resulting data locally. If possible, as long as the security allows. For some data it’s better to use localStorage, for others – sessionStorage, and what does not fit – you can just store in memory. We used Angular so we can use angular-cache
This partly solved the problem – when a page was accessed for the first time, the resources it needed were loaded. Then, the resources were already taken from the cache.
Did, of course, cache invalidation, etc. The traffic decreased, but the initial page response was still unacceptably high.
Background loading of resources
The next logical step was background resource loading. While we are on the main page, looking at messages, alerts, etc – there is a background loading of resources for hidden sections. The idea is that when the user switches to the desired section, the data (or at least part of them) will already be in the cache, the loading is not required, the page response is reduced. The first option is the easiest:
$q.all([Service1.preLoad(), Service2.preLoad(), ...ServiceN.preLoad()]);
Where Service.preLoad() is the function that returns the page resource promise.
But there is a problem – in this case all promises are executed simultaneously, i.e. all 100500 resources are pulled simultaneously. And the channel is not a rubber band! In addition we have a mega-request to third-party service, which was pumping a lot of data slowly and for a long time. As a result, all the parallel requests were taking almost as long to execute as this mega-query.
Okay, let’s download one by one :
It got better, but not very much, because if the user went to the “wrong” partition, we would have to wait for the queue to load until we got to that partition’s resources. All in all, there are endless heuristics here. Something is better to load in batches, something separately in parallel, etc… I wanted, however, some kind of more systematic approach.
Once again, the logical step is to put the loading of all resources into a dynamic queue. Priority. If we enter a partition and its data (or part of it) is still in the queue, we increase their priority and they are loaded first. As a result, response <= than it was. A trifle, but it is pleasant.
That’s true, but there’s one more place that’s not rubbery – the size of the cache.
The deeper I dug into this problem, the more deja vu I felt – I’ve been through this before! Limited memory resources, background loading, unloading, priorities… that’s… that’s the typical game engine resource system! Driving a car – the locations are loaded, what is far behind – unloaded. There was also a special term for game engines – streaming support… I spent 5 years of my life in game mode, it was cool…
So there you go. It turns out that the web application is essentially an analogue of a resource-intensive game. Only here we have not locations, but pages/sections. Pages pull resources and put them into cache, but it is limited in size, we have to unload something. I.e. the analogy is apt.
The problem with a web application is the same as in gamemade – predict where the user will be to load resources ahead of time. The #1 solution is design, of course. Direct the user to a predictable (and sometimes only) route. What design fails to solve – statistics + heuristics.
Unloading is the same. You just need to understand what to unload first. Prioritize resources, unload resources with the lowest priority. Or kill them in piles.
There are a lot of implementation options. The most predictable is preprocessing, either manual or automatic. We have to specify which resources in the location/partition we want, and reduce the priority to some.
And then Ostap got carried away… I mean, gamedev tricks applicable to web development came to mind. One of them is LOD (Level of Detail), a level of detail. In gamdev, it is used for textures and models. You can immediately load the world with the lowest level of detail, and stream already detailed texture models. And the player always sees something, and can even play.
I.e. you need a LOD system for downloaded data! For the web the most primitive variant is suitable – two levels of detail. First we load the initial data which the user sees (the first pages of tables, for example).
The data is small, it loads quickly. And the background… LODs that are “heavier” are loaded with the background.
Cramming the un-crampable in is almost a standard task for a game designer. So let’s push the boundaries of localStorage! Take some LZ-compressor and go! Yes, but localStorage can only store strings… Well, that’ll work then, for example, lz-string.js The compression isn’t the same anymore, but even -20% when you only have 5Mb at your disposal isn’t bad at all! And as a bonus, the securitization stuff, the localStorage will have Chinese characters instead of plain text.
Next… further thought rushes to unknown depths. VFS (Virtual File System), a layer between the resource system of the game and the file system of the operating system, comes to mind. Usually, everything revolves around a data file which can be referred to as a file system. Read the file, write to it… What about a VRC (Virtual REST Call)? Then we could support operation of the web application even if there is no internet connection at all! To a certain extent, of course, but still.
The controllers communicate with the resource manager. What it can give, it gives immediately, all other requests are given to VRC. And it, in its turn, synchronizes its state with the backend and informs about it as it is loaded.
When they talk about the offline operation of a web application, it always slips in Meteor Cool, sure, but we were in a tight development stack. The proposed variant, on the other hand, can be implemented on almost any framework. With reservations, of course, but you can.
But that’s not what this article is about. It’s about how, sometimes unexpectedly, the experience of long ago comes to light…
Have fun coding, friends!