I have just noticed that it's been months since we've talked together. It's been months since we posted a video and I want to take some time now to record a series of videos where I'm gonna explain what we're doing and also tell you about all the amazing things we've been building with dfuse.
Today I want to talk about one of the great things you can do with dfuse and it is streaming table rows, table deltas. So the dfuse team built a custom state database that is able to give you a snapshot of the state of the blockchain at any block height. So that means if you're interested in "what were the orders open in that DEX, at that point in time?" you can get a snapshot and get all the rows in a consistent view. This is very important because the EOS blockchain moves very fast, 0.5 seconds. And it means rows can change.
So if you have a table with a thousand rows and you're querying them a hundred at a time It might happen that you've queried the first chunk, second chunk, and then it moved. That row changed up here. But you know, you're down there querying the rest because you're querying between the block boundaries. So it's difficult to get a consistent view of a table. With the state database that dfuse is offering, you can query on a block height and it sort of freezes the blockchain for you at that point and it will give you a full list. Even though there's a hundred thousand rows, or if it's going to download 100 megabytes… If you want the full view of the table at that point in time, dfuse will give it to you.
So that's one thing. And then you can also take from that snapshot time only the changes going forward. So you're really applying only the 2 or 3 rows that are changing. And if you apply them on top of your initial snapshot, you can have on your side, in your Python code or whatever, a consistent view of what's happening on the blockchain right now. And if you receive a delta you could for example sum it up, or do an operation on the full table that you have in memory. And that makes it so that you're always in sync with the latest truth of the blockchain. And not only that. The stream - that's the function "get_table_rows" here - will also navigate forks. So maybe you want to check the videos about microforks. We'll link it in the description.
But basically it means that sometimes you can have an operation, a transaction, that passes, but it's sort of reverted. Or reapplied by another producer because there were some slight network hiccups. It means that you've received some information that is no longer true. It was true a moment ago, but now it has been reverted. So you want to know that, you don't want to keep that information in your memory if it's not true anymore. So the dfuse platform will stream you the fact that this has been reverted so that you can apply it in reverse, and then you'll receive the next changes which might include that change again because it was just passed on from one Producer to another. But if you follow that stream, that a linear stream, you will be always in sync with what is in the blockchain memory right now.
I hope this is cool, I hope this is helpful, I know it is because people assume that of databases. That they're having a consistent view. But I think if you're not using the dfuse platform you might have surprises. So take a look at "get_table_rows" and also the state snapshot endpoints that you can query with the REST api, or directly in the WebSocket stream to get a full snapshot. And start immediately after with never missing a beat.