Questions about Peerunity gitian release process

Projects
1

This was originally directed to Glv, in response to his pull request for a gitian build process for Peerunity (https://github.com/Peerunity/Peerunity/pull/48) but Glv suggested that I post my questions here so that others may benefit from the discussion.

I don’t consider myself to be extremely proficient with Git and even less so with Gitian. My questions deal specifically with the first part of the release process (https://github.com/Peerunity/Peerunity/blob/master/doc/release-process.txt) …

* update translations (ping tcatm on IRC for now)

* update (commit) version in sources
  bitcoin-qt.pro
  src/version.h
  share/setup.nsi
  doc/README*

* tag version in git

   git tag -a v0.5.1

* write release notes. git shortlog helps a lot:

   git shortlog --no-merges v0.5.0..

* perform gitian builds

  * From a directory containing the bitcoin source, gitian-builder and gitian.sigs
   export SIGNER=(your gitian key, ie bluematt, sipa, etc)
   export VERSION=0.5.1

  1. What part of the above steps are necessary if all I’m trying to do is test the process?

  2. Does it matter which signer is used and how do you create new signers?

Thanks,

pb

2

If you only want to test the build process, you can ignore the instructions for writing the release notes, for updating the version numbers in the source code and for signing the final result of the build process.

First you set up the linux environment, the virtual machines, etc… (described in the file contrib/gitian-descriptors/README)
Then you download the dependencies and build them (described in the file doc/release-process.txt)

Then, as the updated gitian descriptors are not yet in the official git repository with an associated version tag, you will need to do it in a personal copy of the repository:

  • Set up a peerunity git repository containing the updated gitian descriptors (and the associated code changes) where you want
  • Tag this repository with a dummy version number
git tag v1.2.3
  • Add a parameter to the gbuild command to build peerunity to override the url contained in the gitian decriptor (official repository) with your custom repository
bin/gbuild --commit peerunity=v1.2.3 --url peerunity="url of your custom repository" ../peerunity/contrib/gitian-descriptors/gitian-linux.yml

When you’re not testing but building the official version, you sign the result of the build process with your private GPG/PGP key. It allows everyone to verify (with your public key) that you validate the binary.
Gitian is a deterministic build process, so if someone else builds the software, they should get exactly the same result.
So by checking that all the signatures match the same build result, the people can get sure that no developer has added some stuff in the final build without the others knowing.

PS: I’m not an expert on gitian, I started learning how it works a week ago, so if some gitian guru is there, feel free to join the discussion and correct eventual mistakes…

3

Thank you for the explanation. Very helpful!

I’ve got all the dependencies built, all of your changes pushed to my personal repo, and I’ve tagged the test version (git tag -a v1.2.3) but I keep getting stuck here when attempting to build …

$ bin/gbuild --commit peerunity=v1.2.3 --url peerunity="https://github.com/pennybreak/peerunity" ../peerunity/contrib/gitian-descriptors/gitian-linux.yml

fatal: ambiguous argument 'v1.2.3': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
bin/gbuild:219:in `block in <main>': error looking up commit for tag v1.2.3 (RuntimeError)
	from bin/gbuild:204:in `each'
	from bin/gbuild:204:in `<main>'
4

Tags aren’t pushed automatically to the remote repository, so you must do it by hand.
You can push a tag with the same command you use to push a branch:

git push origin v1.2.3

Another solution is to use directly your local repository in the gbuild command:

bin/gbuild --commit peerunity=v1.2.3 --url peerunity="file:///home/USERNAME/peerunity" ../peerunity/contrib/gitian-descriptors/gitian-linux.yml
5

Thanks! I have successfully built both packages and updated my comments in the GitHub issue.

Is your “gitian-peershares” branch ready for testing?

6

Yes, you can test the gitian-peershares branch.
And as you have already built the dependencies to build Peerunity, you can skip the dependency building phase and build Peershares directly.

EDIT: Correction: You don’t have to rebuild boost and Qt, but you still have to build peershares-deps.

7

I decided to rebuild everything on the peershares branch just to thorough. Boost-win.yml is giving me some trouble …

./bin/gbuild ../peershares/contrib/gitian-descriptors/boost-win.yml
--- Building for precise amd64 ---
Stopping target if it is up
Making a new image copy
Formatting 'target-precise-amd64.qcow2', fmt=qcow2 size=11811160064 backing_file='base-precise-amd64.qcow2' encryption=off cluster_size=65536 lazy_refcounts=off 
Starting target
Checking if target is up
Preparing build environment
setarch: x86_64: Unrecognized architecture
./bin/gbuild:21:in `system!': failed to run on-target setarch x86_64 bash < target-bin/init-build.sh (RuntimeError)
	from ./bin/gbuild:76:in `build_one_configuration'
	from ./bin/gbuild:233:in `block (2 levels) in <main>'
	from ./bin/gbuild:228:in `each'
	from ./bin/gbuild:228:in `block in <main>'
	from ./bin/gbuild:226:in `each'
	from ./bin/gbuild:226:in `<main>'
8

That’s a strange error.
The “boost-win.yml” file is the same in Peerunity and in Peershares, and it doesn’t use anything from the source code of Peerunity of Peershares, so if it worked with Peerunity it should work with Peershares.

I just tried again with the same command:

[quote=“pennybreaker, post:7, topic:2387”] ./bin/gbuild ../peershares/contrib/gitian-descriptors/boost-win.yml [/quote]
and it worked fine…

[quote=“pennybreaker, post:7, topic:2387”] setarch: x86_64: Unrecognized architecture [/quote]
The “setarch” command would fail like that on a 32bit system, but why would your ‘base-precise-amd64.qcow2’ virtual machine contain a i386 system ?

9

That was strange. Reboot of the host fixed it.

10

Everything looks good with the Peershares build processes.

Can you show me how to go about including other files in the install process? For example; including a conf file and having the installer place it in the correct path.

11

I’ll take the “gitian-win.yml” descriptor as example for how to add a file to the installer.

In the descriptor, the part following “script: |” is a bash shell script containing the commands that will be used to compile the software and copy the required files and binaries to the final output directory.
The output directory is “$OUTPUT”. Everything you copy to the “$OUTPUT” directory, you will find in the “gitian-builder/build/out” directory of the host after the building process.

For example, after compiling “peerunity.exe”, we copy it to “$BINDIR/”, which is a shortcut for “$OUTPUT/32/” or “$OUTPUT/64/” depending on the target architecture.

cp -f release/peerunity.exe $BINDIR/

The sources of Peerunity are in the directory “$HOME/build/peerunity”.
So to copy a “peerunity.conf” file that would be in the “share/” directory of the sources, you could add in the script:

cp "$HOME/build/peerunity/share/peerunity.conf" "$OUTPUT"

Now, the installer is generated from the “setup.nsi” file (in the “share/” directory).
First, we copy all the sources and binaries in the same directory (to make things simpler), then we generate the installer.

mkdir nsis
cd nsis
tar -xf $HOME/build/peerunity/$DISTNAME
mkdir release
cp $BINDIR/peerunity.exe release/
cp $BINDIR/peerunityd.exe release/
makensis share/setup.nsi

To add the “peerunity.conf” file to the installer, we must add rules in the “Installer” (Main SEC0000) and “Uninstaller” (un.Main UNSEC0000) sections in the “setup.nsi” file.
In the “Installer” section:

SetOutPath $INSTDIR
File peerunity.conf

And in the “Uninstaller” section:

Delete /REBOOTOK $INSTDIR\peerunity.conf

Here the “$INSTDIR” directory points to “$PROGRAMFILES/Peerunity/” or “$PROGRAMFILES64/Peerunity/”.
There are other builtin directory variables, like “$APPDATA”, “$LOCALAPPDATA”, “$DOCUMENTS”, etc… You can find a list of these variables here.

12

Thank you for taking the time to explain all of this!

pb