#254 Allen Day: Google's Mission to Provide Open Datasets for Public Blockchains

https://ift.tt/eA8V8J

Public blockchains produce enormous amounts of data. In theory, anyone can access the raw contents of transaction and blocks. In practice, however, querying blockchains can prove to be a daunting task.

The difficulty lies in the fact that blockchains are particular types of distributed databases and thus carry several limitations. Most, if not all, blockchains lack the most basic SQL querying capabilities supported by nearly every off-the-shelf database system.

Take Bitcoin as an example. Its API lacks even the most basic calls which would allow a user to query any address and receive the balance. In order to achieve this, block explorers and alike have developed sophisticated middleware infrastructure that parses the blockchain, normalizes the data, and stores it in a database, where it can be queried. In the best of cases, companies offer API calls for only a limited set of operations. Google hopes to change this by freeing blockchain datasets.

We're joined by Allen Day, Science Advocate at Google's Singapore office. Earlier this year, he and his team released the Bitcoin blockchain as a public dataset in Big Query, Google big data IaaS offering. In August, they added Ethereum to their list of freely available public datasets, which includes US census data, cannabis genomes, and the entirety of Reddit and Github. Anyone wishing to query the data can do so in SQL on the Big Query website or via an API. For instance, a relatively simple query would return the daily mean transaction fees since the Genesis Block in just a few seconds.

Coupled with Google's AI and Machine Learning infrastructure and other open data sets, one can only imagine the potentially groundbreaking insights we could gain from this data.

Topics discussed in this episode:

• Allen's background as a geneticist
• The similarities between blockchains and evolution process in lifeforms
• Google's cloud platform and its various components
• Big Query and its publicly available datasets
• The Bitcoin and Ethereum datasets in Big Query
• Why this data is useful to the public and for what it may be used
• The particular challenges in implementing Ethereum as opposed to Bitcoin
• Insights we may gain by crossing blockchain dataset with other data
• How machine learning and AI could help us better understand specific transaction patterns

Links mentioned in this episode:

• Bitcoin in BigQuery: blockchain analytics on public data

• Bitcoin Blockchain Public Dataset

• Ethereum in BigQuery: a Public Dataset for smart contract analytics

• Ethereum in BigQuery: how we built this dataset

• Ethereum Blockchain Public Dataset

• Change Agent by Daniel Suarez

• Real-time Ethereum Notifications for Everyone for Free

• ethjs-abi library, compiled for use in Google BigQuery

• Kaggle: Your Home for Data Science

• The Strange Inevitability of Evolution - Issue 20: Creativity - Nautilus

• Google Cloud

Sponsors:

• DutchX: The open, decentralized trading protocol for ERC20 tokens using the Dutch auction mechanism
• Azure: Deploy enterprise-ready consortium blockchain networks that scale in just a few clicks

Support the show, consider donating:

• BTC: 1CD83r9EzFinDNWwmRW4ssgCbhsM5bxXwg (https://epicenter.tv/tipbtc)
• BCC: 1M4dvWxjL5N9WniNtatKtxW7RcGV73TQTd (http://epicenter.tv/tipbch)
• ETH: 0x8cdb49ca5103Ce06717C4daBBFD4857183f50935 (https://epicenter.tv/tipeth)

This episode is also available on :

• Epicenter.tv
• YouTube
• Souncloud

Watch or listen, Epicenter is available wherever you get your podcasts.

Epicenter is hosted by Brian Fabian Crain, Sƒbastien Couture, Meher Roy & Sunny Aggarwal.

via The Let's Talk Bitcoin Network https://letstalkbitcoin.com/blog/post/epicenter-254-allen-day-googles-mission-to-provide-open-datasets-for-public-blockchains