๐๏ธArchitecture
Massbit Route Architecture Overview
Last updated
Massbit Route Architecture Overview
Last updated
With the booms of NFT games/decentralized applications, and the surge of developers beginning to work with Layer 1 networks, many projects seek for third-party toolsets to connect their apps to desired blockchains. The majority of existing dApps are utilizing blockchain Infrastructure-as-a-Service (IaaS) such as Infura or GetBlock to shorten the time to bring their products into production. Those blockchain IaaS services have gained traction quickly and contributed to the growth of many Layer 1 and 2 blockchain networks.
But that added value comes with a downsize, many "dApps" are relying on centralized blockchain IaaS services. Amazon Web Service reports more than 25% of Ethereum nodes are hosted on their compute platform, which is fully controlled and managed by the company.
In September 2021, Amazon Web Services experienced an outage in its US-East-1 region that affected numerous websites, from Disney+ to VICE's, as well as its own warehouse operation. Caught in the incident was dYdX exchange, a large trading volume "decentralized" exchange for Ethereum derivatives. dYdX services and websites experienced serious latency and impaired functionalities.
There are still some parts of the Web3 infrastructure relying on centralized services, and Massbit Route is deeply committed to a fully decentralized future of the Internet.
With the current third-party blockchain IaaS services, your dApps data is being transferred to a centralized cluster of blockchain nodes. This means your dApps are prone to outages occurring in the underlying infrastructure that hosts third-party blockchain nodes.
Massbit Route is a blockchain distribution network (BDN) operated and governed by users all over the globe. It provides decentralized applications access to different blockchain APIs by routing request network traffic to a blockchain node that has optimized response time. In addition, Massbit Route is built to provide performance-optimized access to blockchains and aims at eliminating blockchain API single-point of failure by forming a global network of independent Node Providers.
We build a network operated and serviced by users that provide fast and redundant access to blockchain sources. Massbit Route facilitates a network of Gateways and Nodes infrastructure in 6 different zones around the world:
Asia
North America
Africa
Europe
South America
Oceana
Users can join Massbit network as Providers from any of those regions by running Gateway and Nodes that are connected to blockchain data-sources such as Ethereum or Polkadot. As Massbit network grows in the number of Nodes and Gateways, the network becomes highly available and redundant. When a Provider experience an issue with their nodes, blockchain API requests will be served by other Providers, which eliminates single-point-of-failure in Web2 CDN architecture.
In Massbit network, Gateway Manager, Community Nodes and Community Gateways form a global blockchain CDN network in order to optimize blockchain API request and response time.
Based on the public IP of each community-run Node and Gateway, a global geographic map of verified and staked Gateways is continuously updated. When a dApp sends a blockchain API request through Massbit Route API Entrypoint created by a Consumer, based on the global network IP map, the request is forwarded to a Gateway in which its zone is the same or closed to the request source IP.
Each Gateway constantly updates Massbit Core to maintain a list of verified Node in its zone. From the gateway, the request is then forwarded to one of the nodes in the same zone, which will then proxy the request to the blockchain data-source associated with the node.
When a node or gateway experiences issues such as poor CPU, memory, and network performance which result in high latency and response time, it will be deregistered from Massbit network by Fisherman. Only nodes with "staked" status in Massbit network can serve API requests and earn token rewards to guarantee the stability of the entire network.
Because Massbit Gateways, Nodes, and blockchain RPC nodes are operated by independent providers, single-point of failure is eliminated. Fisherman ensures DNS entries of offline or malfunctioned Nodes and Gateways are removed from the network and make sure blockchain requests are forwarded through active nodes, which maintains redundancy for Massbit network.
The domain portion of the dAPI URL is a dynamic record. When the caller (DApp) send a request to the dAPI for the first time, it need to resolve the to the IP address of an active Massbit Gateway by sending DNS lookup requests to the Gateway Manager. The resolved IP returned to the caller is the IP of a nearby Gateway that can serve blockchain RPC requests quickly to the caller.
In order to find the closest Gateway to the caller, Gateway Manager needs to have an idea about the caller's location by looking at the Source IP. Gateway Manager will then look up the geo-location (country and continent) of the Source IP from MaxMind database, then return the IP of the Gateway that is in the same Country as the Caller.
Most Web3 or DApp users will lookup DNS from their ISP's DNS servers or public DNS servers. When those DNS servers do not know the answer for DNS queries for Massbit dAPI, they will forward those queries upstream until they reach the authoritative DNS server which is Gateway Manager.
By design, every hop in the DNS chain terminates the connection and initiates a new one toward the next hop. Most of the Web3/DApp users have their DNS server set to their ISP's DNS or common public DNS servers. Since most queries come from intermediate Recursive Resolvers, the source address is that of the Recursive Resolver rather than of the query originator. Many ISPs use a Centralized Resolver infrastructure that can be distant from the clients the resolvers serve. In these cases, the Gateway Manager may get the wrong idea about the user location, which in turn may lead to a poor Gateway selection.
EDNS is the newer standard the packet size for DNS above 512 bytes to support location extension. However, many public DNS servers are still servicing with the traditional DNS standard, which affects the Massbit Gateway selection process.
To mitigate the Recursive problem and help improve the quality of Massbit dAPI as well as user experience, DApps has an option to use the Redirection approach.
By adding /_redirect or /_getlink to the HTTP dAPI URL, Massbit Route will return another URL that points to a more accurate Gateway group. Specifically, when the caller's HTTPS/WSS request first arrives at a Massbit Gateway, the Gateway looks at the Source IP of the caller in the request's layer 3 header and determines whether the caller's request has reached the correct Gateway.
In case the request reached the wrong Gateway due to the Recursive DNS problem:
If the dAPI URL has /_redirect in its path: the Gateway will send back a 308 HTTP_PERMANENTLY_REDIRECT Response with the URL that points to the group of Gateway in the same Geo-location as the caller. Applications that implement the RFC-7538 should redirect automatically after receiving the 308 response.
If the dAPI has /_getlink in its path: the Gateway will send back an HTTP response with an URL that points to the group of Gateways in the same geo-location as the caller. This method is suitable for DApps that do not implement the RFC-7538 standard and want
to handle the redirection on their own. In addition, applications that use Web Socket connection should also utilize this method to obtain the URL that leads to the correct Gateway group.
The new redirect URL has the following format:
Once the DApp obtains the new URL, it also needs to perform DNS lookup with the Gateway Manager. This time, the URL indicates the exact geo-location (Zone/Continent and Country) of the caller, thus the Gateway Manager can resolve the domain name to one of the Gateways in that region.
When a Node or Gateway joins Massbit, it needs to go through different states before it can serve dAPI traffics.
Created: Node/Gateway information (name, zone, blockchain type that is served by this node, blockchain node IP) is registered with Massbit network. Based on the registered information, an installation shell script is generated for the new node.
Installing: When the installation script is executed on a new server that needs to join Massbit network, the status changes to Installing, and the public IP of the Node/Gateway is recorded to Massbit chain
Verifying: When the installation completes on the new server, it will proceed with the verification process for eligibility to join Massbit network. Fisherman will be mainly responsible for this stage by checking the following criteria:
Data integrity
The node/gateway must return data identical to the blockchain data-source from which the response was retrieved. This will prevent man-in-the-middle attacks
The blockchain data-source must be fully synchronized.
Performance
Node/gateway needs to satisfy 95% of 1000 RPC requests/sec with the max response time of 500ms
Verified / Failed: If a node/gateway passes Verifying stage, it becomes "Verified" and is ready to be approved by Fisherman for staking and serving dAPI.
Fisherman will
Approved: In each zone, based on the dAPI demand, Fisherman automatically will approve the node/gateway to join Massbit.
Staked: After the node/gateway is approved, the Node Provider needs to stake a minimum of 100 MBT tokens. Once the node/gateway is staked, it is ready to service dAPI requests and earn token rewards.
