Programming language of the future
The IT industry changes faster than any other industry, due to the flexibility of how information is exchanged and processed. And so do the languages, with which programmers express their intentions to a computer. But it is my impression, that the development in this space becomes more and more like searching for a local minimum, in a space constrained by previously adopted abstractions, and the limitations that come from them. There is nothing wrong with that, but what interests me is - can we have revolutionary progress, instead of evolutionary? If it is possible, it would take questioning the assumptions made until this point, challenging our very viewpoint. So, let us engage in some critical thinking. Image source Let's examine the most general type of program. It takes requests, usually in the form of HTTP requests, processes them on stateless agents behind a loadbalancer, and stores the output in a database. Instead of raw HTTP, user is presented with a web interface - made with Javascript, the agents could be servers, VMs, containers in K8s or a serverless environment. The database is SQL until greater scalability is required, after which NoSQL alternatives are resorted to. Image source This is a distributed system - but it sure isn't programmed like one. The "distributed" component is "solved" by the database - the rest of the system stays local. This results in wasted opportunities, and a painful mismatch between technologies. Image source Let's talk about "systems" or "backend" programming languages. The last revolution in this space was the C language - all progress since then was iterative. And in large part this has been a failure - VMs, containers, Nix - they are only necessary, because the core system is broken. Programming languages implement their own memory management, like in most modern languages, their own schedulers, like in any language with coroutines or virtual threads, their own state management - because the database that the OS provides for you is the filesystem. The OS doesn't provide the right abstractions, or provides them poorly. And if your language tries to "abstract away" from the underlying OS, and do things its own way - you get mismatches and unaccounted for corner cases, like for a web server that runs as one process, when it runs out of memory due to an outliar request - the whole server crashes. This is a sad state of affairs overall. Image source The response to all this has been the growing popularity of Javascript, and the browser as a platform. This, again, shows the failure of all "native" solutions - how hard is it to draw a window? If you know anything about X11 and it's open-source implementation Xorg or Wayland and its open-source implementation wlroots, quite hard. Which is why a document viewer, meant for reading text files that link to other textfiles, won in the end. Maybe ChromeOS is the future - only once we consolidate the technologies at the top of the tech stack, can we switch out technologies and abstractions at the bottom. But alas, we are where we are. Image source And Javascript has room for growth itself. The browser is still there primarily for interactive documents - so when creating an application, one has to fight with the browser. You need either immediate-mode rendering, with Flutter, or a good baked-in reactivity system - which the browser doesn't provide. This is for client-side. For server-side, NodeJS creates the dominant paradigm - based around the event loop, which breakes the abstraction of a Unix process. Furthermore, there is no way to gracefully handle OOM - you either let the program crash, or set arbitrary limits on the number of concurrent requests, and hope there aren't many outliars. But even with that, the technological mismatch between JS in the browser, and C-like languages on the backend makes Javascript on the server a viable option. I don't think it's a good option - just that the alternative is even worse. Image source And lastly, the database. A technological marvel, meant to make different programs changing state feel like "this" program is the only one running - like the OS. Both fail at the job, both are a bad abstraction. What you get in the end is MVCC that presents different versions of reality to different programs, optimistic locking based on what state has the transaction seen so far, two phase commit, meant to make distributed programming feel local. If databases are good, why isn't it industry practice to expose a public schema as a read-only API, and only use GET requests for rendering actual documents? Image source But concurrent mutation of state is only half of the problem - distributed state is the other half. In the 90s the industry attempted to solve this, by the means of distributed objects - and got nowhere. They wanted to solve the general problem - distributed programming - by using specific tools, meant for a specific problem - local-only programming. You can solve the spe

The IT industry changes faster than any other industry, due to the flexibility of how information is exchanged and processed. And so do the languages, with which programmers express their intentions to a computer. But it is my impression, that the development in this space becomes more and more like searching for a local minimum, in a space constrained by previously adopted abstractions, and the limitations that come from them. There is nothing wrong with that, but what interests me is - can we have revolutionary progress, instead of evolutionary? If it is possible, it would take questioning the assumptions made until this point, challenging our very viewpoint. So, let us engage in some critical thinking.
Let's examine the most general type of program. It takes requests, usually in the form of HTTP requests, processes them on stateless agents behind a loadbalancer, and stores the output in a database. Instead of raw HTTP, user is presented with a web interface - made with Javascript, the agents could be servers, VMs, containers in K8s or a serverless environment. The database is SQL until greater scalability is required, after which NoSQL alternatives are resorted to.
This is a distributed system - but it sure isn't programmed like one. The "distributed" component is "solved" by the database - the rest of the system stays local. This results in wasted opportunities, and a painful mismatch between technologies.
Let's talk about "systems" or "backend" programming languages. The last revolution in this space was the C language - all progress since then was iterative. And in large part this has been a failure - VMs, containers, Nix - they are only necessary, because the core system is broken. Programming languages implement their own memory management, like in most modern languages, their own schedulers, like in any language with coroutines or virtual threads, their own state management - because the database that the OS provides for you is the filesystem. The OS doesn't provide the right abstractions, or provides them poorly. And if your language tries to "abstract away" from the underlying OS, and do things its own way - you get mismatches and unaccounted for corner cases, like for a web server that runs as one process, when it runs out of memory due to an outliar request - the whole server crashes. This is a sad state of affairs overall.
The response to all this has been the growing popularity of Javascript, and the browser as a platform. This, again, shows the failure of all "native" solutions - how hard is it to draw a window? If you know anything about X11 and it's open-source implementation Xorg or Wayland and its open-source implementation wlroots, quite hard. Which is why a document viewer, meant for reading text files that link to other textfiles, won in the end. Maybe ChromeOS is the future - only once we consolidate the technologies at the top of the tech stack, can we switch out technologies and abstractions at the bottom. But alas, we are where we are.
And Javascript has room for growth itself. The browser is still there primarily for interactive documents - so when creating an application, one has to fight with the browser. You need either immediate-mode rendering, with Flutter, or a good baked-in reactivity system - which the browser doesn't provide. This is for client-side. For server-side, NodeJS creates the dominant paradigm - based around the event loop, which breakes the abstraction of a Unix process. Furthermore, there is no way to gracefully handle OOM - you either let the program crash, or set arbitrary limits on the number of concurrent requests, and hope there aren't many outliars. But even with that, the technological mismatch between JS in the browser, and C-like languages on the backend makes Javascript on the server a viable option. I don't think it's a good option - just that the alternative is even worse.
And lastly, the database. A technological marvel, meant to make different programs changing state feel like "this" program is the only one running - like the OS. Both fail at the job, both are a bad abstraction. What you get in the end is MVCC that presents different versions of reality to different programs, optimistic locking based on what state has the transaction seen so far, two phase commit, meant to make distributed programming feel local. If databases are good, why isn't it industry practice to expose a public schema as a read-only API, and only use GET requests for rendering actual documents?
But concurrent mutation of state is only half of the problem - distributed state is the other half. In the 90s the industry attempted to solve this, by the means of distributed objects - and got nowhere. They wanted to solve the general problem - distributed programming - by using specific tools, meant for a specific problem - local-only programming. You can solve the specific problem with general tools - however poorly, but not the other way around - you always need to start with the general problem. So now the industry resorts to the saga pattern and frameworks like Temporal - which, again, shows the inadequacy of the core technology. Wouldn't it be wonderful if the language allowed us to express distributed state without friction?
All of this leads us to where we are now. And I believe we can do better.