OP_RETURN coming to Peercoin's blockchain

I don’t if we discussed OP_RETURN in such detail, if so, please point to the already existing post.

So I see Sunny Officially declared:

[quote=“Sunny King, post:1, topic:3473”]Weekly Update #139

[ul][li]OP_RETURN small data transaction support is now added to peercoin. This feature is requested by the peermessage project and will be included in the coming release.[/li][/ul]

Have fun![/quote]

I am now wondering if:

,that means we’ll have 80 new bytes added to every transaction that wants to include OP_RETURN (or does that mean “every” transaction automatically has 80 bytes more?

I also found this written by theymos

http://bitcoin.stackexchange.com/questions/29554/explanation-of-what-an-op-return-transaction-looks-like

An important aspect of OP_RETURN is that outputs which use it in the standard way are provable unspendable. This means that nodes can immediately remove such outputs from their unspent outputs cache and potentially forget about them altogether (though Bitcoin Core doesn't do this yet). This makes OP_RETURN transactions much less expensive for the network than other ways of stuffing data into the block chain

So I am hoping for this:

Will the Peercoin implementation (either now or in the future) allow these OP_RETURN to not be stored for years and years? Or a way to prune stale ones out of the blockchain after they’ve aged?

The intent is not to prune them out of the blockchain, but to prune them out of the unspent unspent transaction output set that resides in memory (since by definition an op_return is provably unspendable).

Only transactions with op_return outputs have this 80 byte space. Other transactions still take up the amount of space they took up before (frequently more than 80 bytes). Fees are still determined by number of bytes in a transaction, and op_returns pay the same fees as any other transaction.

I wonder if it would be worthwhile to introduce an OP_RETURN FLAG that is 1 byte, which basically means that the OP_RETURN can be pruned after 520 blocks at a later date.

Your PeerMessage OP_RETURN should only need to exist for 520 blocks? How about 1024 blocks? Not forever, forever ?

I believe there will eventually be blockchain pruning opportunities in the future. It’s just a matter of finding, calculating, and compacting transaction data.

Right now we don’t have to worry, because the chain is small enough. But in 30 years, it’s going to accumulate a lot of data that is really not important. For instance…

Pretend Peercoin was invented 30 years ago, in 1985.

Other than for nostalgia reasons, would we really need to know a transaction that was sent so long ago?

When planning for the future. Act like you’re looking at the past. It helps. Where were you in 1985, and if you fired up your Peercoin client in 2015 and saw it downloading data from 1985 before you could sync, how would that make you feel.

How much disk space from 1985 would you like to allocate?

But if some one showed up and said, oh no, you can totally prune out certain flagged transactions in the blockchain. Then all of a sudden a 1985 OP_RETURN could be pruned out of the chain…

That’s 30 years of 80 byte “flagged” OP_RETURNS. Could be a big deal.

Putting all arguments about Moore’s law and increasing internet bandwidth speeds aside (they have been rehashed incessantly by bitcoin folk about op_return), your underlying point is valid and I have no objection to it.

Everything I intend on using op_return for within the peerapps network could be pruned from the blockchain after period of time with minimal ill effects to the peerapps network. A lengthy period of time would be ideal (say 1-2 years).

I suspect that’s not a problem we need to solve now though. We can sit back on our small blockchain and reap the benefits that are found from other blockchains wherein blockchain size is their main problem, and they need to focus significant effort on various pruning solutions/strategies.

I expect options for blockchain pruning (like this: http://cryptonite.info/wiki/index.php) will be more widespread in the future.

It’s absolutely worth investigating, but as Emeth mentioned, it’s not necessarily something that is critically needed right now.

Cool. I like that idea. I agree.

You may be misunderstanding.

All I’m asking is for 1 extra byte for OP_RETURN_FLAG which could mean:

0 - Permanent OP_RETURN (may not be pruned)
1- ShortTerm OP_RETURN (may be pruned after 520 blocks = 3.6 days)
2 - LongTerm OP_RETURN (may be pruned after 105200 blocks = 2 years)
3 - reserved for future use
4 - reserved for future use

…Some thing like that. By default, for PeerMessage, you could be setting his OP_RETURN with a flag of 2

Some one might come along in the future and want to set a Flag code of 4 for some future use without having to fork the network to do it.

I think this 1 extra flag byte is something we should consider now.

WE DON"T EVEN NEED TO PRUNE TODAY. Just flag them as “prunable”, and then yes, we can worry about it later.

I don’t understand how we can agree to 80 extra bytes, but not 1 extra byte to pre build in some responsibility before we start filling our chain with extra 80 byte transactions.

This laid back approach of Moore’s Law, and “We’ll worry about it later” is irresponsible and not very visionary.

The time to do this is NOW. Before OP_RETURNS start showing up. Otherwise you have the problem of having to figure out which OP_RETURN later can be pruned, and which one can’t. That’s building problems INTO the system from the start, without the flag.

My view on the issue at the moment:

  1. There is no way to prevent ppl from stuffing data to the blockchain if they insist. This is somewhat due to script system (one might argue that even without the script system data can still be encoded in address or send amount). The only real discouragement of data on blockchain is the transaction fee. I am talking about general principles here, I have nothing against peerapps using the feature on peercoin. The idea is so long as fee is paid, it’s considered okay. And users of this feature is actually being graceful to the network by not encoding data in obfuscated ways.

  2. From bitcoin/peercoin core point of view, these probably can be pruned immediately from unspent set. Note pruning does not remove it from blocks. It’s still on the blockchain and can be retrieved from blockchain at any moment albeit with higher access cost.

  3. I would be very cautious modifying the bitcoin script system, due to the issue that maintaining such modification can be expensive long term. So one needs strong argument why its benefit is so great.

  4. The proposed system of declaring different expiration to the network isn’t quite consistent with my view that transaction fee is the only guard on data flood. It kinda suggests that fee alone is not enough, that additionally the network still expects you to be ‘nice’ and let nodes prune you more quickly.

For the record I want to see old records kept. I want to see transactions happened in ancient Egypt if they were recorded in a blockchain. :))
Micro etch the blockchain in stone if needed.
Better yet, put it in a primecoin block chain. Primecoin protocol might be understood across the Galaxy.

Here is an important aspect of blockchain technology that shouldn’t be taken lightly. I’m talking far off into the future, but this is some thing that visionaries now should be prepared for…

Even if this becomes an issue after we’re all dead, and future generations need to deal with it.

Blockchain pruning and reorg will eventually happen. If you plan for storage space, speed, and computers of the future to handle blockchain size (Moores Law) it is irresponsible in the early stages of designing Peercoin to hope that works out.

None of us know how many transactions will happen in the future if this becomes adopted worldwide. None of us know how large blockchains will grow.

An example:

Some one mined Peercoin early on, generated 500,000 peercoin, and even shows up in the forum and admits, yes, I was a whale, I had 500,000 peercoin, but my hard drive died, I had no backup, and those 500,000 peercoin never minted a block as a result. That 1/2 million peercoin are now lost. I will not use them, they will never be recovered, they are dead.

Now fast forward 100 years (yes, all of us reading this are dead too), our grandchildren’s children are dealing with the blockchain in some new ways that would blow our minds.

Why couldn’t the protocol burn those 500,000 peercoin and re-distribute them into the system?

Now at this point, we were talking about OP_RETURN. I chose this example to lead with, so you could see how there will be future reasons to burn wallets, OP_RETURNS, etc, to slim the blockchain at a future date.

The novelty in all of this, is being awake and aware of the future even though none of us see a need for it now. What it does do, is build longevity into the system.

All of these forked PoS coins are so worried about what will sell today, or be useful today. Since Peercoin is the mother of these PoS implementations, I think it is only prudent that we take a different approach and build into our blockchain responsibility, especially when we add new data to the chain on the next fork.

This is a subject that needs attention now, and needs to be an issue ongoing as Peercoin evolves and develops. If we’re the ones building a “green” version of low energy, low space utilization, then my thoughts on this are not strange.

mhps wants old records kept. That’s fine, old records can be kept, perhaps on a side chain, or on an official “archive” chain.

But I will tell you, in 100 years from now, some PeerMessages sent, where the PeerMessage developer tells you himself, that his OP_RETURNS will probably be no longer necessary on the official chain in 1-2 years needs to be heard loudly.

In 99 years from now, we don’t need that PeerMessage OP_RETURN. All I asked for is one a 1 byte flag.

How is it that we an happily introduce 80 bytes for OP_RETURN, but not a 1 byte flag?

By the way, on my first example, about the 500,000 whale address that could be burned. I believe that if a wallet holds coin, generates 0 stake after 5 years, does 0 transactions after 5 years, we should be able to burn that wallet balance and redistribute it to minters or miners.

A wallet balance without minting or mining, or moving, doesn’t need to live on the blockchain for 10,000 years. It’s obviously a rogue balance that needs to be pruned or burned.

All of what I’m proposing seems like a non-issue since what we have is working and suits our needs today.

To be a sincere pioneer in the industry though, these are issues that need to be addressed in the early stages. We should not leave problems for future generations if we can avoid it.

[size=14pt]My last example is spam and email. [/size] Early internet RFC’s didn’t think about abuse of email and spam. No realization of abuse of email was properly written into early RFC’s, and now you know why spam is as bad as it is… It’s a plague. Their attitude was “we don’t see it, and if it does happen, then future can deal with spam and email”.

Well we still haven’t solved this spam problem properly either (filtering minimizes but doesn’t solve it). Email authentication should have happened from the start. It didn’t.

Peercoin is still very new, and its technology has been adopted numerous times via multiple forked coins. Let’s put some real thought into this, so if Peercoin outlasts my life time or yours, we knew we did the best we could to prepare for future generations.

Here’s the immediate benefits:

  1. We’ll be unique to Bitcoin, because we’re planning for the future even more than they are

  2. We’ll prove ourselves as a real backbone, since we’ll continue to minimize our chain. Any one can add more bytes to their chain for this feature, or that option. But to be able to be self-pruning, or self-burning later, because we took precautions now means we’ll thank ourselves later.

  3. PPCMAN is asking for a 1 byte flag. Heck, it can even be an “optional” flag, that no one must abide by. But at least give sidechain developers who use OP_RETURN a chance to set a 1 byte flag if they want.

Here’s the long-term benefits

  1. You’re building into the system flags, that can be quite useful later for future use, or to better understand the new data flowing into OP_RETURN once we fork.

  2. MOST IMPORTANT If you don’t flag from the introduction of OP_RETURN, you can’t go back to yesterday and start flagging. This is why I’m begging for the 1 byte flag to exist. If new developers don’t bother to set a flag, or go with the default flag, then it’s totally on them for being irresponsible. At least that functionality was available to them at the time.

Finally, picture this:

Disk defragmentation tools for hard drives. Ever see a visual representation of a hard drive utility defragmenting a disk? Think of Peercoin what it might need 100 years from now. A 1 byte flag with multiple uses is needed.

If I know any thing about Sunny King, he doesn’t like to hard fork very often. So if you’re going to hard fork for OP_RETURN, then please add a flag. It doesn’t need to be called OP_RETURN_FLAG, call it even OP_FLAG.

[size=14pt]At the end of the day, it’s Sunny King’s decision. At least I had him consider it. I will sleep well tonight knowing we discussed this before the fork.[/size]

[quote=“ppcman, post:9, topic:3474”]… I believe that if a wallet holds coin, generates 0 stake after 5 years, does 0 transactions after 5 years, we should be able to burn that wallet balance and redistribute it to minters or miners.

A wallet balance without minting or mining, or moving, doesn’t need to live on the blockchain for 10,000 years. It’s obviously a rogue balance that needs to be pruned or burned.[/quote]

I cannot disagree with the above quote more strongly. I believe in the long term integrity of funds held in Peercoin. Who is to say this is a “rogue balance” and not a brain wallet long term investment held by Hal Finney in cryo-suspension? Burning other people’s money because it has not been used for some time is a horrible idea. If I thought this idea had any merit in Peercoin, then I would sell all my Peercoin and go in search of a better long term crypto.

I agree
It s a scary thought that the algorithm that defines the properties of ppc can be changed in the future into something else. Last time the same thought occurred was the suggestion the 1% reward ought to be increased… :frowning:

+1

In my point of view (ppcsuite developer) we better stay as close as possible to Bitcoin implementation in order to benefit from Bitcoin ecosystem and network effect.

[quote=“NewMoneyEra, post:10, topic:3474”][quote=“ppcman, post:9, topic:3474”]… I believe that if a wallet holds coin, generates 0 stake after 5 years, does 0 transactions after 5 years, we should be able to burn that wallet balance and redistribute it to minters or miners.

A wallet balance without minting or mining, or moving, doesn’t need to live on the blockchain for 10,000 years. It’s obviously a rogue balance that needs to be pruned or burned.[/quote]

I cannot disagree with the above quote more strongly. I believe in the long term integrity of funds held in Peercoin. Who is to say this is a “rogue balance” and not a brain wallet long term investment held by Hal Finney in cryo-suspension? Burning other people’s money because it has not been used for some time is a horrible idea. If I thought this idea had any merit in Peercoin, then I would sell all my Peercoin and go in search of a better long term crypto.[/quote]

I understand the knee jerk reaction, where it seems this is wrong. But I’m thinking far ahead. Perhaps 5 years isn’t enough. Let’s say 25 years, where the wallet holder has died, buried and is 6 feet under in a cemetery.

At some point blockchains (irregardless of Moore’s law) are still going to be bloated and certain data is going to be unnecessary. Every one (all coins) are currently building problems into their implementations today that will be discovered in the future. Yes, I know we have light clients, etc, but still. Eventually blockchains don’t always need to start from genesis. There has to be a new approach to this… Let’s think outside of the box, not only because we can, but because it is our due diligence to do that.

Blockchain technology is inherently flawed because pruning and burning isn’t part of the design, and it should be… We’re innocently creating problems now for our children’s children, etc.

Every one is so free to lower blocktimes (thankfully peercoin is 10 minutes while other new coins try to have 30 second block times)… the amount of data that it creates is a progressive problem even worse than PoW.

So while every one likes to +1 a disagreement to my concern, my concern is still valid.

The short answer of that, is yes, that serves us today, tomorrow, but not 5 years from now. At what point does Peercoin no longer rely on Bitcoin’s way of thinking and really fork or branch off on its own.

In my opinion, Sunny King has built peercoin not only to show that Proof of Stake can work, but knowing all along that Bitcoin’s system is flawed. In my opinion he knew that eventually all the gazillion alt-coins would eventually have to consolidate and seek out a backbone currency. Today it’s Bitcoin. I expect in our lifetimes it will be Peercoin or some thing very similar.

Creating an altcoin is free. Pumping and dumping an altcoin is somewhat free.

The whole cryptocoin industry is fragmented beyond belief and needs to find a common ground (backbone blockchain) to build from.

To give an example, if every one wanted to create their own internet, and gain adoption, do you know how much of a wasted effort that would be? From the beginning you’d be building an impossibility unless you came up with certain consensual standards.

Luckily we have RFC’s (Request for Comments) that build in “standards” on the internet today. HTTP is a standard. SMTP is a standard. I expect that with the right adoption and effort PEERCOIN could be a standard in it’s own light.

Once the industry realizes we’re padding a row boat with wooden toothpicks, we’ll soon come to a consensus and learn to build side chains to a mutually agreed upon backbone.

We’re going no where fast. We’re inevitably going to discover the need for a consentual backbone currency. Bitcoin is fulfilling that need for the moment, but I don’t believe it can forever fill that task. Peercoin is way better suited for it.

I don’t know how long it will take the public to have that “aha!” moment, but Peercoin will be here as a solution ready and waiting.

With all of that being said… I still hold behind my sincere feeling that before we introduce more data to our blockchain, the responsible thing to do is to design pruning and burning flags to re-optimize it.

It sounds too scarey when you say it. OMG ppcman, so what your saying, is that if I don’t mint, or transact for a period of 6 months, 2 years, or 5 years, my balance dies?

NO, I’m not suggesting that. really (even though I used it as an example to spawn discussion).

At this point, all I asked is for a 1 byte flag, and this is the work I have to go through to convince people that we need to build in responsibility into our system.

I have no idea why I stand alone in this issue. Is building responsibility into our blockchain that freakish?

[size=10pt]DO NOT IMPLEMENT OP_RETURN WITHOUT AN OP_RETURN 1 BYTE FLAG TO DECIDE HOW LONG IT LIVES ON THE CHAIN[/size]

P.S. If Bitcoin is going to have the wherewithal to be responsible, this might actually be a PULL request from Peercoin.

…and for those of us lazy programmers, we don’t need to implement the pruning OP_RETURNS today. But when we’re ready, at least the flags will be there. Even the PeerMessage app developer admitted, 2 years of his OP_RETURN could be pruned

ppcman, I respect you and do not mean to offend you.

You disagree with the way Bitcoin (and now Peercoin) have implemented op_return as a mechanism for storing data.

Thus far, you have been unable to convince another developer to code it the way you prefer.

Your options are:

[ol][li]Let it be.[/li]
[li]Continue trying to convince a developer to code it the way you prefer.[/li]
[li]Code it yourself, and submit a pull request.

[list]
[li]If Sunny accepts your pull request, your change will be added to peercoin core![/li]
[li]If Sunny does not, you can either try to get 51% of the community to swap to your modded client, or fork the coin.[/li]
[/list]

[/li][/ol]

I sympathize with some of the points you made, but this is an open source community, not a democracy. When consensus cannot be achieved on vision, the decision will be made by developers who ship rather than armchair philosophers. I encourage you to submit a pull request for consideration.

I hope that even if things do not go the way you want that it doesn’t affect your involvement in this community - we are far better off having you here!

#shipit

I appreciate that, and I respect and do not mean to offend you either.

Yes, I do, because storing data on a blockchain, that doesn’t need to exist forever, should have a flag mechanism when it can be pruned and deleted.

It’s early yet. Don’t sell my eggs before they’ve hatched. There is a significant value I bring to the table about this issue. Just because someone hasn’t coded them into a commit doesn’t mean there isn’t a need for it.

[quote=“emeth, post:15, topic:3474”]Your options are:

[ol][li]Let it be.[/li]
[li]Continue trying to convince a developer to code it the way you prefer.[/li]
[li]Code it yourself, and submit a pull request.
[list]
[li]If Sunny accepts your pull request, your change will be added to peercoin core![/li]
[li]If Sunny does not, you can either try to get 51% of the community to swap to your modded client, or fork the coin.[/li]
[/list]

[/li][/ol][/quote]

Yes, those options seemingly seem accurate today. My pinpoint in this discussion is Sunny King himself who has decided to import OP_RETURN into Peercoin’s code with his latest update.

The thing here is that I’ve noticed over the last couple of years, Sunny King has always done “what’s right” and “what might not be popular at the time”…

When Proof of Work seemed popular, he still came out with “Proof of Stake”, because it was the right thing to do

Also, there were Bitcoin features and enhancements, and new developments that never made it into Peercoin core right at the beginning, because I believe there was no rush and Sunny King was patient enough to see the results of that code before daring to import it into Peercoin.

Finally, you have Sunny King doing development on two major achievements in cryptocurrency:

  1. Proof of Stake, which has given birth to hundreds of forked clone adoptions

  2. Prime Numbers, the first crypto coin that did that… and broke World Records more than once

Looking at Sunny King’s history, and last 2 years worth of work. He’s a very diligent coder. A patient person, and a visionary in the things he creates, whether or not the price market depends on it.

Admittedly, I’ve private messaged Sunny King before about “if we don’t do this now, the price could suffer as a result”, and he never blinked an eye. He continues to sign his messages “Have Fun”.

Sunny King isn’t propelled by market price of his creations. He’s propelled by doing what needs to be done for the future. He’s not just a visionary, he wants to leave a legacy. I agree with that mantra. Some of us are here in this world to facilitate the creation of things that we may not enjoy in our life time. It’s a way of giving back (or paying it forward) even though we may not be rewarded today.

Actually it is a democracy even though it doesn’t seem like it is… If popular consensus speaks loudly enough, and Sunny King agrees with the consensus, then it is only natural that he would want to support the consensus.

Proof of that is cold wallets and minting. It was spoken loudly, a consensus was reached in these forums for the need, and we have in our future a solution to the problem.

I’m going to smile while you make that analogy. I’m not am armchair philosopher even though I could easily be pegged as much. I found Peercoin only because of the importance of what it brings… I continue to be a part of Peercoin not because of the coins I hold (it’s pitiful less than <$200 cuz of real life issues), but because of what I can bring to it too.

If you want to split hairs in the community between C++ coders and every one else is an armchair philosopher… then that’s a very narrow minded view. Simply because you have the ability to use github and make C++ commits and I can’t.

I’m similar in ideologies of what Sunny King has… let’s have fun and see what we can do as a community. I participate because I feel what I do from the heart is necessary, irregardless of negative criticisms .

[quote=“emeth, post:15, topic:3474”]I hope that even if things do not go the way you want that it doesn’t affect your involvement in this community - we are far better off having you here!

#shipit[/quote]

I echo that statement with you to emeth. In fact, any one that participates in PeercoinTalk forums that has any thing to say… we’re all far better off…

With all of that being said.

I still firmly stand behind the need of this:

IF YOU EXPAND THE BLOCKCHAIN FOR OP_RETURN. THEN PLEASE ADD A 1 BYTE FLAG TO DECIDE WHEN WE MAY PURGE IT.

ppcman, I’ve been following your comments and I believe you’ve made your points very clear.

I have a few thoughts of my own which I would ask you to consider.

Firstly, although your intention is toward a smaller blockchain the immediate impact of your proposal is obviously a very small incremental increase. Your end goal relies upon some future time when a consensus may or may not be reached regarding a pruning feature. Thus, it is possible that your proposal does nothing but increase the data size indefinitely.

Secondly, there is no incentive for users to ever elect to have their data flagged for pruning. Why risk losing access to historical information when the cost to an individual for keeping it forever is virtually equivalent to trashing it after only a few days or years?

Thirdly, I believe that you too easily dismiss the role of light clients in the future for the wide majority of regular users. While I may not be able (or desire) to store the entire Peercoin blockchain on my cell phone 99 years in the future, I have no doubt that there will be many audited and reliable data centers around the world capable of storing Peercoin’s blockchain in its entirety from genesis to “Apocalypse.” :wink:

Fouthly, you kind of lose me when you begin talking about burning old (lost?) coins. I’m not sure how you envision this related to OP_RETURN; however it points to probably the biggest concern I have to your proposal. In my opinion, breaking with Bitcoin protocol should only be done under severe cases where there is clear and immediate benefit. Adding an additional byte to OP_RETURN in Peercoin exposes us to an untested vulnerability. Although the additional byte may appear innocuous enough now, it is conceivable that some future protocol change or new client could exploit this extra byte to “prune” other data including UTXOs. When you further add in the benefits of riding on Bitcoin infrastructure, the risk/reward balance is heavily on the side of maintaining symmetry with Bitcoin.

I do appreciate your passion for maintaining the lightest blockchain possible, but I think you should remember that this is one of Sunny’s objectives as well, and he has already designed the fixed transaction fee to contain the storage and bandwidth cost purely through economic forces.

You clearly have foresight and creativity in coming up with this idea; hopefully even if it is not ultimately included in the upcoming protocol change, you’ll still feel satisfied that your concerns were carefully discussed, and I truly hope you’ll continue working with this community to find new ways to improve Peercoin in the safest way possible.

[member=8565]learnmore[/member]: well articulated +1 :slight_smile:

I think the most important here is this:

First pruning doesn’t mean we shrink the blockchain. Some nodes will have to keep prunable data forever, whether there’s a special flag or not. It is required so that new nodes can validate the blockchain. If there’s anything missing in the blockchain you can’t validate it.

When we talk about prunable data we mean that a client can delete these data from his own copy of the blockchain after he used them to validate a block. He won’t need these data to validate future blocks. But then he’s not a complete full node anymore, because he can’t provide the full blockchain to others.

Second, anyone who doesn’t use these prunable data and doesn’t want to store the complete blockchain can delete the prunable data just after he received it. So a flag is not necessary: prunable data already means you can delete it as soon as you want.

This flag could be a hint, suggesting how much time nodes should keep it. But this decision is more a matter of how much they want to be a full node, and has nothing to do with the emitter or the data itself.