Ethereum’s Progress on Developing the New Internet
Updated: Feb 15, 2019
This article will provide a quick background of the need for a fundamentally new Internet architecture and then go on to describe the developments in building a new Internet infrastructure and protocols. It will finally describe the progress of the most advanced project, the Ethereum network, in bringing the promise of a new "Web 3" enabled economy to fruition...
Quick Background
Status Quo: The Internet
Computing networks have evolved significantly in the last 50 years, progressing from local data storage/processing to global networks of data storage/processing connected through Internet protocols. However, the capability that existed in the 1980s merely to access information on another web-connected computer is trivial without an easy way to navigate the vast network of computers that is the Internet for specific purposes.
In the recent era, major Internet-based companies such as Google, Facebook, Amazon, Uber, and AirBnB build platforms that organize information, and connect users with each other, with information, and with products. Their success is resulting in the development of huge, privately owned, highly centralized platform databases so that users end up having to go through these intermediaries to access and interact with information, other users, and products.
These platform networks have transformed society in many ways, with one of the more promising developments being the emergence of a sharing-economy (facilitated by platform networks such as AirBnB) and the gig economy (facilitated by platform networks such as Uber/TaskRabbit/etc). These new economic models allow people to rent/sell goods and services directly to one another very quickly on an as-needed basis and to provide new types of scalable, as-needed employment.
At the same time, there are many service/rental/purchase arrangements that are currently not easily arranged through a digital platform especially where there is little profit for an intermediary to justify the investments in developing a platform. Moreover, several high profile acts of censorship have destroyed these opportunities for people to engage in the sharing/gig economy. (See the Uber ban in London or the Airbnb limits in Europe).
Platforms like Etsy, Amazon, and Alibaba allow individual entrepreneurs and companies alike to reach wide and diverse customer bases around the world. These platforms connect creators, consumers, buyers, and sellers for transactions involving innovative, custom goods and/or competitively priced commodity goods. Again however, these centralized platforms are subject to the whims of national governments who may wish to prohibit products or platforms from competing in their countries. (See Amazon's travails in India)
As recent Congressional hearings have demonstrated, public concerns have become more pronounced that a significant amount of the value of the user/product/business data stored in these centralized databases is captured by the platform owner and the data can be compromised and commercialized in bothersome ways.
Ultimately, the companies that own and operate these platforms are subject to government and commercial pressures so that government intervention or the withdrawal of advertiser dollars may induce these companies to censor activities and information on their platforms (Chinese Internet censoring of search results is one example, and YouTube censoring videos deemed too controversial to earn ad revenue is another example).
Status Quo: Financial & Legal Systems
Whereas the structure of computing networks has changed significantly, the financial and legal systems that facilitate and regulate transactions between these market participants has scarcely changed. For the last 50 years, small numbers of large (in some cases too-big-to-fail) financial institutions (mostly serving nation-states) have been accumulating customer account and transaction information using basic ledger methods and have been transacting amongst each other using private centralized financial telecommunication systems and clearinghouses.
These systems and methods take several days to process and settle, with several intermediate processes, entities, and regulations slowing transactions down even further. This exclusive system of accounting for, securing, and processing financial information has resulted in a small number of firms having tremendous control over the people’s abilities to interact with each other where the interaction involves the exchange of value. (While credit card networks have alleviated some of the burden of commercial transactions, millions around the world are not eligible or do not have access to credit card networks, and these networks cannot handle large international transactions for things like real estate or capital goods).
Additionally, financial transactions, especially international transactions, involve expensive obstacles such as foreign currency exchange risks, inflationary risks, sanctions and other regulatory obstacles. Where countries' central banking institutions control the supply of money available and the prevailing interest rates, transactions, even domestic transactions, are subject to inflationary risks. These same central bank tools can also be used to manipulate currency valuations to the advantage of domestic exporters and to the detriment of international trade. Ultimately these financial risks discourage or at least create drag on the growth of international trade.
Finally, the legal system that enforces and regulates rights and responsibilities involved in commercial agreements has proved to be too expensive, complex and time consuming to engage with for the regular consumer, dealing with small to medium size transactions. Even for large corporations the burdens of commercial litigation for ambiguous contracts have become prohibitively resource intensive, causing most large companies to resort to extrajudicial arbitration systems. International commercial disputes run into an even more burdensome layer of complex jurisdictional issues before the merits of a case can even be examined.
Status Quo in Need of Improvement
Ultimately this combination of a vulnerably centralized state of the Internet, an opaque, self-interested financial system, and a slow moving balkanized legal system impairs much socio-economic and geopolitical development that a more free, open, decentralized marketplace could foster.
Building the New Internet
Blockchain Basics
In 2009, the pseudonymous Satoshi Nakamoto adapted timestamping software methods from the 90s and combined these methods with newer cryptographic methods to establish a new “crypto-economically” viable digital currency.
Basically, users of this cryptocurrency, instead of transferring US dollars or some other fiat currency from their account to someone else’s and waiting for the respective banks to update their ledgers of balances/transactions, use the digital version of ledgers of accounts/transactions themselves as currencies. These ledgers do not reflect account balances and transactions denominated in any national fiat currency, but instead reflect transactions in a new digital currency (in Nakamoto's case, "bitcoin.")
Because every transaction would need to be reflected instantaneously on all copies of the ledger, the ledger becomes an ever-growing chain of blocks of more and more recent updates. Each block of transaction/ledger data is verified by tens of thousands of freelance "miners" around the world who lend their computing power to perform cryptographic verifications in exchange for bitcoin. Verified transactions and ledger updates are then added to the chain of blocks that together form the blockchain currency ledger.
Moreover this chain of ledgers is not stored on a centrally controlled computer (as with most financial institutions that “attempt” to secure their singular copies of ledgers on a handful of private servers) but is continually being updated and managed simultaneously by tens of thousands of computers around the world.
In order to manipulate or alter the blockchain one would need to gain control of tens of thousands of computers in different parts of the world all at the same time (Computers processing or “mining” the blockchain are in countries with wildly different, and in many cases opposing, political systems so its is unlikely that even government could or would be able to sabotage globally distributed blockchain records).
It is this system of globally distributed computing that ensures the security and integrity of the blockchain because gaining control of so many computers in so many parts of the world simultaneously would be nearly impossible.
However, while the concept of cryptocurrency is exciting in that it creates the opportunity for a truly international financial system that facilitates instantaneous transactions between people/businesses anywhere in world with a smartphone without need for currency conversions or banks accounts, or government/other corrupt actor interference, it is trivial compared with the next generation of blockchain innovation: distributed applications.
Distributed Applications
In 2013, a young Bitcoin researcher named Vitalik Buterin, realized that instead of the transaction blocks containing mere transaction information, the blocks could contain application states generated by programs written in a generalized programming language (in addition to cryptocurrency transactions using in this case Ether).
Thus any developer could write a program involving complex computation processes and have the progress and results of the program be captured securely and immutably on the blockchain.
This programming language, in principle, could describe any type of interaction with human, machine, or software participants with outputs that are built into a secure, but accessible blockchain.
For example, a class of programs akin to contractual agreements, known as “smart-contracts” could be built to set up conditional arrangements so that if all or some conditions of an agreement are satisfied, results or payouts (full or partial) are executed automatically, nearly eliminating opportunities for fraud.
Other classes of programs could govern funds with payouts openly accessible to anyone who completes a series of specified tasks. In sum, programs could be developed which automatically execute upon satisfactions of any number or complexity of conditions.
Other programs could facilitate secure social interactions such as secure picture and video sharing and messaging.
In short, there is little done on the conventional Internet that in principle that could not be done more securely, transparently, and profitably (for the parties internal to transactions as opposed to external parties or owners of platforms) on a the new distributed internet. Domestic and/or International transactions and interactions would be regulated according to a disinterested algorithm (which is open to anyone to learn and to program) and executed reliably and instantaneously without the need for banks or courts or government intervention or Internet companies.
So what progress has been made on this kind of distributed computing network...?
Ethereum’s Progress on Developing the New Internet (2018 Year in Review)
The Ethereum platform is attracting more and more developers and researchers to increasingly broad layers of its ecosystem. Though the network and the technology underlying are still in their infancy in terms of their capabilities and even their techno-economic philosophy, the pace and scale of the development of the ecosystem means the network may achieve the critical mass of talent and resources needed to become the underlying protocol of a new distributed internet architecture (sometimes referred to as Web 3).
This accelerated development in the Ethereum network is certainly not reflected in the price performance in 2018 of Ether (the cryptocurrency required to pay for processing resources on the Ethereum network). Nor, is this development yet reflected in the mass adoption of the network for everyday consumer transactions.
However, the broad development over the course of the past year 2018 illustrates the attraction and excitement of developers at the prospect of helping to build a new distributed internet architecture – a vision that has been put forth by the Ethereum foundation even its early years: see their quick video Ethereum: The World Computer (2015)
So what development occurred exactly in 2018?
Web 3 developers/facilitators Josh Stark, Evan Van Ness, and Daniel Zakrisson, in a long article in Medium, describe and summarize all the broad development efforts across the ecosystem.
They draw 6 illustrative conclusions about the networks progress over the course of 2018:
1. Ethereum started to be used at or near maximum capacity
2. Innovative and useful Decentralized finance applications went live
3. Infrastructure improved and Developer tools proliferated and became easier to use
4. Early Scaling technologies began to go live, and research continued on profound scaling solutions
5. Collaborative research on zero-knowledge technologies started suggesting even more profound applications for scaling
6. The development roadmap started to coalesce into more of an engineering effort rather than a research effort
These conclusions are explained below in just enough detail to reach satisfactory understanding:
1. Capacity Utilization
Throughout 2018, users of the Ethereum blockchain network consistently employed almost all (above 80% for most of the year) of the all gas (the finite commodity internal to the network used to incentivize miners to create new blocks in the blockchain) available in the market to pay miners to memorialize their transactions on the blockchain, as can be seen from the below graph:
Even though the number of transactions plateaued as the frenzy of speculative trading drastically declined, the “composition of transactions on Ethereum shifted from a large number of simple ones, to a smaller number of complex ones.”
This is consistent with the emergence of blockchain utilization for capturing more complex application states (the architecture underlying Web 3 functionality), rather than mere cryptocurrency transfers.
Several prominent, more complex applications of this kind went live in 2018 including Augur (a betting and hedging application), decentralized finance applications (see below), gaming applications, and Spankchain, a secure Uber-esque smart contract between age-verified consenting adults that facilitates adult entertainment web-cam transactions. (Though this last application may be distasteful to some, this is a secure, decentralized version of the economic model referred to as the gig economy and/or the sharing economy (think Uber drivers, Task rabbit, Airbnb)) (Also, this secure smart contract ensures payment to vulnerable adult models who traditionally only receive a small percentage of the compensation paid in the traditional adult industry with the middlemen taking large portions).
Though the daily active users who interact with a dapp’s smart-contract on the blockchain only averaged between 10,000–15,000 users on any given day in 2018, this is only because the network, which recall is operating at maximum capacity, does not have the capacity yet to run larger scale consumer applications and currently only hosts applications that facilitate rather narrow markets.
In any case, these new, more complex, and more useful applications created demand for better infrastructure that will in turn “make it possible for the next wave of applications to work at a larger scale.”
2. Decentralized Finance applications
In 2018, “[m]any projects launched which were explicitly financial — applications or protocols that give users new tools with which to manage and use Ethereum-based money or assets. As a group, these became known as ‘Decentralized Finance’ or ‘DeFi’.”
“While these tools are in very early stages, it is possible today to use Ethereum-based protocols to take out loans, to lend money and earn a return, to buy bundles of assets, to hedge your risk, to trade assets trustlessly, and to make payments for zero fees.”
There are a few exciting technologies in this area including lending tools (Dharma, mainnet May 2018, Marble, mainnet beta July 2018), margin trading and derivatives products (Daxia, mainnet January 2018, dYdX, mainnet October 2018, bZx, mainnet September 2018, Market Protocol, testnet November 2018, UMA, under development), bundled investment products (Set Protocol, mainnet June 2018), money market protocols (Compound, mainnet September 2018), credit default swaps (CDx, under development), token swap services (Kyber Network, mainnet February 2018), subscription payment services (8x, testnet October 2018), payment channel hubs (Connext, mainnet September 2018), stablecoins (Dai, mainnet December 2017), and prediction markets (Augur, mainnet July 2018, Gnosis PM, alpha December 2017).”
“Why did DeFi take off in 2018? One reason is that many of these applications are useful today, even before critical scaling technologies are in place. Basic financial use-cases like lending and borrowing do not require high transaction throughput — they simply require a secure programmable base layer blockchain. Ethereum’s simplest use-case is the creation, exchange, and use of digital assets like ETH. One way of looking at DeFi is that it is simply building the basic financial infrastructure to use those digital assets.”
3. Infrastructure Improvements and Developer tool proliferation
Application Development tools
Importantly, in 2018, thousands of application developers increasingly downloaded the most popular Ethereum blockchain application development platform, Truffle Suite, in order to begin creating new decentralized applications.
In addition, new alternatives and competitors to popular decentralized application code development tools became available. The Stark et al. article describes the importance of the wider adoption of these tools: “In 2018, the vision of average developers being able to build something useful on Ethereum became a reality, and the tooling necessary to use smart contracts in production improved.”
Security
Moreover, new security tools, that provided security analysis were released and the Ethereum security community began to share “best practices” through resources like the Smart Contract Weakness Registry. In addition, “notable ‘traditional’ security researches bean working in the Ethereum spaces including Trail of Bits and Sigma Prime, adding to the stable of high-quality auditing firms already working in the space.” This latter development is a sign of the growing respect the wider software community has for the Ethereum network’s progress and potential transformative capabilities.
Infrastructure
The two most popular portals/clients to access and interact directly with the Ethereum blockchain, Geth and Parity, “continued to be improved and refined,” and new portals/clients were released. In addition, more projects started to provide developers looking to host their applications on/access the full blockchain network more convenient, less hardware intensive options.
More importantly for wider adoption, more platforms and protocols were developed to allow easier access to blockchain applications and to information stored on blockchain networks through everyday web browsers.
With respect to wider infrastructure development, “[t]he global community of Ethereum researchers & developers got better at working with each other in 2018,” with more regularly-scheduled, widely attended teleconferences on myriad specialized subjects.
Finally, in an illustration of the growing developer-friendliness of tools and platforms, more and more major Ethereum conferences were able host hackathons, where competitors participate in 36-hour races to build working demos in response to interesting challenges and fun prompts.
4. Scalability research and development
The reality is that coordinating the processing and storage of the entirety of blockchain application states and transaction data results in massively unwieldy layers of webs of data, which ultimately limits the rate of transactions the blockchain can store. Thus, researchers and developers made significant progress in developing mechanisms to “off-load computation from Ethereum onto ‘off chain’ systems, while still retaining a blockchain’s characteristic security guarantees. These off-chain systems can process transactions faster and more efficiently than the Ethereum main-chain, leading to more scalable payments or smart-contracts.”
The most basic solution, state channels, allows multiple transactions between parties to be conducted off-chain for a certain amount of time before the final state of their accounts is added to the blockchain. Many prominent projects went line in 2018 using payment channels to facilitate faster and cheaper transactions. Moreover, the framework for generalized state channels was released as open soure code to make it easier for developers to build channelized applications. Counterfacutal has even launched the Counterfactual Playground platform to allow developers to test their channelized applications.
The next widely anticipated scaling technique is Plasma, in which operations are moved off-chain like in state channels, except that the off chain transactions are themselves captured in a secondary blockchain, “where they can be performed faster and at lower cost,” and users retain the ability to “withdraw their assets to main-chain, even if the operator of that Plasma chain tried to censor or steal from them.” Research into this area advanced in 2018 and basic implementations of payment-only sidechains have been developed.
5. Zero Knowledge technology application scope developed
The concept of Zero Knowledge proofs “let[s] us prove that some operation happened, without having to share the underlying data. This concept in blockchain circles began as a potential privacy solution, but in 2018, was described concretely as a blockchain technology in a seminal whitepaper ZKSARKS and also re imagined as a potential scaling solution as well.
The scaling implications come into play because if the zero knowledge mechanism perform its confidential-transaction verification, “cheaply enough, then it could let Ethereum smart-contracts verify that an operation took place off-chain. This means that we could, for instance, conduct large numbers of operations off-chain, and then cheaply verify that they happened. Or, we could conduct intensive computation off-chain, and still have it verified on-chain.”
Many libraries for zero knowledge proof protocols were compiled and many projects began to work to fully realize the promise of zero knowledge technology
6. More consensus around Research roadmap and development priorities
For a few years now, Ethereum researchers have been focused on fixing the problems of the concentration of transaction processing power that plagues the Bitcoin community as well as the scaling problem that has been drastically limiting Ethereum’s usefulness for large scale consumer applications. In recent years, they have developed fundamental base-layer upgrades to address both problems: Proof of Stake to resolve computing power concentration and Sharding to resolve scaling issues.
Proof of stake is a change in the mining incentive system wherein miners are no longer rewarded for pooling together hugely energy intensive mining operations (or "farms" as their called) where hundreds of thousands of specialized computer processors conduct cryptographic proofs to add to the blockchain. Instead miners are paid in proportion to the amount of their own Ether that they stake so that there is little incentive to tie up large pools of currency in order to gain nominal returns. Mining would be done by smaller freelance miners.
Sharding reduces the computational slowness that results from having to work with a block chain that contains every transactions ever done by breaking up the blockchain into "shards" that are then processed simultaneously but in some coordination and communication.
The mathematical and conceptual foundations for these solutions have become firmer and the 2018 roadmap evolved from a research and design agenda to an engineering agenda.