Bitcoin.com Mining Team - What can miners do to improve instant transactions?

[Music] so one thing that is that we can point out is that mining pools or most other mining pools probably don't really care that much about instant transactions the mining pools job is to put transactions in blocks and instant transactions happen outside that process so they it's not really their priority to to think about these kind of things as especially if they're only a mining pool so what let's go from let's go through some through some of the things so miners do honor first seen first safe room by default if they are running an unmodified client most miners do run unmodified unmodified clients but as Peter showed here on Tuesday there are some miners that seems to have modified their clients to actually do RBF even though that's not activated by default in in any of the the currents become cash clients which is quite what's quite a quite interesting find and there's little incentive actually modify your client especially since we're doing Network wide upgrades every six months it just leaves a lot of extra work for the miner and most miners don't want that extra work so a double spend succeeds when a mining node sees a double spending transaction before the merchant node because the Koston miners do honor first seen first safe that's actually what we saw in Peters presentation on Tuesday is that if you if you just send the the double spend transaction if you sent it almost at exact time or even at the exact time you don't really know which one has got to get mine because they honored first scene for a safe and - prioritize sinking block headers and blocks and sinking their mempool ziz not drill a priority and since it's not being done by default in any of the current of the clients right now it's not really it's not really happening so so yeah let's go through some mining pool trivia so one misconception about mining pools is that it's just one big system and it's very smart at selecting transactions mining pools are not yes one server and they're not very smart and so like a big mining the mining pool one a place we want to place our mining pool service as close to the hash rate as possible which means that most mining pools they I have a lot of sites in North America in Europe and in China and they they don't really have anything in like continents where no one is mining like in Australia so a big mining pool may consist of twenty to forty different mining nodes across the world and one thing that is that people keep for like some people keep forgetting is that the like if you if one pool has forty servers spread across the world technically these are forty different mining pools there you have tagged the blocks the same and they have to collect the data in like in the same central server but from a mining perspective these are separate pools that can orphan each other and which is which is something that we see quite often of mining pools can orphan themselves and by default there's no way to actually synchronize the men pools between the different mining nodes unless Bitcoin D has a feature for it because that would required a mining pool to make the mining pool software do it and there's currently not a very strong incentive to do so since the mining pool mostly us focus on pushing out the block as soon as possible as soon as is found so a transaction that was accepted by varieties in California can be double spent by the same mining pool in China if the transaction was not propagated fast enough and like another thing that about my nipples tell most of them are hosted in the same cloud providers in China all of them are at Alabama I like BTC calm and and pool and by b2c and lay there all hosted on the same cloud infrastructure so if you want to be as close to the mining pools as possible you should just set up servers these cloud providers I think here all of you here have noted basic so if they mining pool but since this will be in YouTube later and now I guess we've run through the basics so one misconception we saw it in the previous budget presentation is that ant miners don't build blocks it's the mining pool software that builds the blocks the ant miners they all do the hashing so the pool soft were asked for a block template from Bitcoin D the RPC and then the pool software converts it into the stratum job the stratum job is sent to the ant miner the hasher the a minor set finds a valid block sends it back and sends back a few variables to the pool and then the pool construct a complete block and then submits it a Bitcoin D so if you really want to like visualize how mining works it's actually the mining pool software where the magic happens the the Hatcher itself is a very very dumb unit that makes a lot of silly mistakes and incorrect shares so I like the setup pretty looks like this you have the Bitcoin these are connected to each other but the mining pool software most of them are not like we started a few mining different mining pool software when we built our our own mining pool and the norm is that there's very little synchronization between the pool software itself at least when it comes to transactions and Men pools most of the inter pool communication actually happens on the Bitcoin on the Bitcoin network itself between the Bitcoin DS so in this example I made the the pulsar for different colors they could be three different pools or they can be the same pool but the fact is that these three pools will compete with each other and will offer an each other even though if they are the same pool or not they don't really care and they're pretty pretty dumb because the like they just work on a block template it comes from Bitcoin D so everything is dependent on the information that comes from the Satoshi client so if I improve mental synchronization all of that has to be done in Bitcoin D by the the client developers I observed a few question for the past few a few questions for the past few days why would the miners honor the first seen first safe rule and the question is pretty simple miners are lazy they just wanted one to default clients modifying the the Satoshi client means that you have to actually get into the code and do your own modification need to recompile everything and that could lead to risk so most mining pools don't really want to take that risks we did see some evidence that it seems to be some miners that actually have taken that risk and implement RBF on their own on the network but most of them they do honor the the first thing first days for a safe robe simply because that's the default setting which is why RBF is the thing on core because like the court Peter taught tried to make the the miners run it without the patch being in core but they didn't really care until us became default in core and the miners didn't really bother removing it so the most of the norm is just run the default default settings and also what is an honest miner it seems that that that's not really clear and my definition I would say an honest miner is someone that runs the default settings without much modification because you have so you have the default setting and then you have what's allowed in consensus so you can operate within consensus but outside the default settings that's where you find our behalf for example as you were not the orphaned because you do RBF but it's still like is it honest and I would say an honour hard say but like probably the best definition is like an honest miner is a predictable miner it means that other minor that you have predictability and how they're going to act because you know exactly what kind of rules they follow I think Josh you asked why the pools don't publish nerman polls it's because we have too many servers so there's not one man pool they're like 30 of them spread across the world and they're not very well synchronized so we can't really publish the nipples is there a way that you know I know there's multiple servers but they could have you know nodes that are not the node that's doing get blocked template but connected one hop away and then expose those nodes mempool so and then it would be up to the people checking to check cross multiple pool servers because really you're just looking for anything distances right you're looking for double spends so it doesn't have to be a singular location it could definitely be multiple locations yeah well personally like I don't really I don't really think that it will be a like any problem publish them and pool but I would like to synchronize it internally first and we're gonna get there I think it will be more healthy and as the transaction count goes up on a network mining pools will find that they will need to sink their men pools better because I mean block block propagation also gets better so how can we sink the the mentals better the easiest least the solution is probably weak blocks where utilized compact expand or graphene graphene being the the better solution here for weak blocks to indicate which block is being worked on and the first step is to do this internally the problem is a weak block don't really help decide the victor in a case of conflict which leads us into pre consensus territory because like you can do this internally inside a pool pretty easily because you trust your own notes so if you have a so if you have a node that so if you have a big mining pool you have service in China in the US and you observe a double spend and then you you can easily make your nodes agree on which transaction to go for simply by sharing the timestamp and you can trust the timestamp because it's your own pool and it's your own servers it gets a little more tricky when it's with an external pool that you have to trust because you don't necessarily know if they're gonna be honest or not if they can if they're gonna modify their time stamps but at least you would be aware of that something fishy is going on which probably is good enough for for merchants so I would say like my ideal solution was be some kind of mimic some kind of internal BGP functionality to decide like which conflicting transaction to go for and if you can old but you can also do some sort of bgp peering between pools so basically like you have a list of week blocks that you receive and you see like which pools like how many transaction they have in their blocks and what's the difference and you can request retransmission of certain transaction and once in a while also observe conflict and how to how to resolve this conflict I think that's that will be a long discussion because like the the avalanche protocol that we just heard about like speaking as a mining pool a mining pool would be pretty upset if if we can get our blocks orphaned by mistake or if there's some kind of arbitrary rules that we cannot predict so the thing is since we are honoring first scene for safe so it's all one is if someone is broadcasting a transaction at the exact same time in China in the US they have the exact same timestamp almost and we try to honor a first scene because the Chinese pool saw it first and the US pool saw the air there one first like how are we actually going to decide which one to go for and if the pool feel that there was no way to actually predict this properly and they got the work they could have the block orphan just because of that the the miners will not accept this protocol they - want predictability they need to be able to know exactly what's going to happen if they get block gets accept you're not so how do we know a which transaction that will be mined well the easiest way is to simulate what might impose does so any merchants that want to know like you can never know of course it's impossible to know for sure 100% that if recession will be mind but ask the merchants you can actually simulate your own mining pool by setting up service across the world so in this case you just set up full notes that are spread across the world on different ISPs different providers with bu and ABC and XT and what to do is that you query each node at exactly the same time and you ask them to do a get block templates and check the transaction you're looking for if the transaction is that everyone's block templates all across the world even though they're spread out and using different clients then it can be pretty sure that it will be mined in the next block as a merchant you can feel not 100 percent safe it gives you best in every case but you can feel 99.99% safe that this will actually be mined in the next block I say this is we call it like transaction confidence the yes query different nodes across the world and if they all agree that they will mine your construction in the next block then you can predict that yeah this is probably not gonna be a double spent and for a merchant that should be enough because that's how the mining pools work basically like this Rock template to solve it and they they they push it out so I would recommend the merchants that want to want to be more safe about instant transactions and see if will be included in next block just simulate your mining pool set up a few notes and distribute them equally across the world especially where the mining where the mining pools are hosted which is male versus North America our Virginia Oregon California some parts of Canada China of course and you yeah Europe eastern and northern Europe and Iceland any questions um yeah I have a question I think I've gathered that from your talk but maybe as a confirmation do you mean that the different pools basically within your pool also use defaults Bitcoin the block synchronization messages currently methods the current methods for for their block synchronization or is there anything to make that faster or more efficient for these by D yeah they use the defaults of course but most mining pools including ours we have special techniques for pushing out the block headers faster than normal would be on the on the block we're using the the Bitcoin network because it's not really it's not that fast and it's faster to do it with custom software but what we're only like pushing out things when we mined a block when actually already found it yeah but is it like when you you mean specialized software that just change the network layer or is there really a different network for that to pass them these these headers so our mining pool and I know that seek a pool and vibey to see or have like customized software to us to replicate or broadcast a block across the world as fast as possible that but they also use the the the Bitcoin layer because you do submit block to Bitcoin D and when that happens you will send a compact block or an extent all the knows you're connected to so it's basically it's like firing a shotgun it's like you just try to fire as many bullets and as many darkness as possible okay thank you um so you said that there is there is nothing much in place to reconcile the members of across your different notes around the world and sometimes one note can actually orphan the other I was just wondering like you still have to reconcile the state of the shares from your from your underlying minors right so you said infrastructure like not reusable it has a different characteristic um that that's not usable at all because I sherry just indicated that a minor actually solved a small block or basically a share means that a my the hasher solve a weak lock and then that just recorded somewhere but for for block propagation and synchronize and making sure that like the mem poles are synchronized and like deciding on which double spend to to go for that's not really relevant it says the difference separate issue could you comment as a as a mining pool on topology how many nodes the mining pool connects to are is each section connecting to the network on their own is there any sort of information so most mining pools I guess they they have like a list of nodes to connect to so like actually like a lot of their mining pools can appeal with each other and mining pools are probably the the mining nodes are actually the most the best connected nodes on the network it's hundreds and they focus on like connecting to the other mining pools of course another thing that I wanted that I wanted to point out that I forgot is that technically the term mining node is not really something that exists because Bitcoin D doesn't mind anymore it used to do that in the CPU mining days but now it's actually like it's a Bitcoin D node with mining pool software installed on the same server they can do it probably you're probably not gonna find a block yeah hi so uh thanks for the great talk and I actually have a question you said that internal you can often yourself and then you know a view that you have to fast blockhead I realize so did you think about like basically making like a trusted internal way to say okay there's this blockhead I'm going to do SP baby mine on top or do you not want to go there because of like risk of attacks somewhere um most so somewhere I do know that some mining pools are better at orphanage itself than others so that for example like I don't think that gonna calm has ever orphaned is itself say really you know if we have I don't think we have because like we make sure to to do everything to prevent that and I do know that my B to C does the same I think and pool is pretty good at orphan in itself and slush pool oh yeah yes thanks for the great talk Emily I actually learned learned a lot about mining that I didn't didn't know I didn't know what that poles morphing themselves I always thought there was just one main pool server so my question I am a proponent of wheat blocks as a lot of people know and one of my concerns with rolling out week blocks was I thought in order to get an adopted you know you'd have to have buy-in from a lot of miners because I thought it only really be useful if the most network was using it but based on your talk it sounds like even if B you were to implement a production version of wheat blocks it would be useful for Bitcoin comm just to synchronize the metals of your own eternal service yeah yeah so even if we just one mining pool uses it is still very useful and it might even be the case that you that the weak flux will only be used internally and they don't really because they would use they would need to set up peers with other mining pools and which they might not want to do but it would definitely be very useful internally and like I'm from a nice P background so like I've been working a lot with PGP so like I would definitely like I see it as sort of like when you run like show IP bgp peer native neighbors on a router you'll see a list of like your neighbors which would mean which would be the other mining pools and what kind of block they're currently working on that would be very useful as a mining pool thank you so there was an RPC code that was added in Bitcoin D a wall by call Prius block and it's so the way it works is that in case there is a race between two blocks the node software is gonna choose the Prius block if you ask it to yeah and I was I was curious to know if that's used by miner a lot or not right because the goal would be the different between deed that are run by a miner when one of them find a block it can call precious block and the others and and make sure that you don't often yourself from the my it from the other money I am from the other my new pool software that I have studied I haven't really seen that in other mining pool softwares yeah no okay yeah so this is a little bit different than the presentation that you had but my curiosity is if there was improvements to mining pool software and the way mining is done today what recommendations do you have for developers that want to assist mining pools and miners to develop better software and what pain points do you see today that could be improved one of the biggest pain point pain points right now it's the peer-to-peer network between the the Bitcoin notes it's it's dreadful it's pretty slow and we should definitely explore using like UDP for example you can use protocol buffers to communicate between the Bitcoin nodes and I that will be s be an improvement in general for the whole Bitcoin network like the natural layer is pretty slow and of course it is slow because for years it was stuck at one megabyte so no one really had an incentive to actually improve it thank you okay thank you very much [Music]