12:00 <@dholbach> Hello everybody! We'll move on to Packaging 101 now. If you have any questions, can you PLEASE move them to #ubuntu-classroom-chat and add "QUESTION: " to them, to make them easier readable 12:01 <@dholbach> I'll have a look every now and then and answer them, if they refer to what I'm currently talking about, if not we can deal with them later. 12:01 <@dholbach> If you want to become part of the MOTU team, or help out with packaging in general, apart from knowing how to write or debug code, it's good to know the tools the distribution uses. 12:01 < Adri2000> hi :) 12:01 <@dholbach> This session won't make you a packaging expert, but afterwards you will have heard of some things, which will make it easier to find your way around. There'll be a MOTU session on Wednesday, 15.00 UTC and Saturday on 16.00 UTC, which will cover what the MOTU team does in more detail. 12:02 < xeruno> why isn't this channel moderated? 12:02 <@dholbach> Please run the following command, it will take some time to work in the background. 12:02 <@dholbach> sudo apt-get install pbuilder; sudo pbuilder create 12:02 < _tx> hi 12:03 <@dholbach> rmjb: MOTUs are the "Master of the Universe", the team of maintainers that takes care of the package in Universe and Multiverse - that's where you start as a package maintainer. 12:03 <@dholbach> rmjb: http://wiki.ubuntu.com/MOTU 12:03 <@dholbach> Packaging - what does that involve? 12:03 <@dholbach> * adding enough information to the Upstream source, to make it buildable on a minimal system. 12:03 <@dholbach> * split the installed files up into separate packages depending on the target audience 12:03 < xeruno> dholbach, could you please moderate this channel? 12:04 <@dholbach> ^- example everybody uses libgtk2.0-0, but not everybody has an interest in libgtk2.0-dev 12:04 < apokryphos> with +z would be better 12:04 <@dholbach> come on guys, that's not necessary at the moment 12:04 <@dholbach> move chatter and discussion to #ubuntu-classroom-chat please 12:04 < jpetso> xeruno: it worked before, i think it's ok to trust everyone on being quiet 12:05 <@dholbach> * make the packages work out of the box 12:05 <@dholbach> * add copyright information, nice description, documentation, etc. 12:05 <@dholbach> Package maintenance involves far more than that, Jordan Mantha (Laser_away) will give a talk about that later today at 21.00 UTC and Thursday at 20.00 UTC. 12:05 <@dholbach> First we'll install some tools we'll need for the session: 12:05 <@dholbach> sudo apt-get install devscripts dpkg-dev build-essential 12:06 < _tx> what about complying programs in Ubuntu to be able to work on all the platforms (ppc, x86, and x86_64) 12:06 <@dholbach> I also prepared an example package we'll use also: 12:06 <@dholbach> mkdir 101; cd 101; wget http://daniel.holba.ch/temp/packaging101.tar.gz 12:06 < jonh_wendell> dholbach: what does pbuilder create do? 12:06 <@dholbach> jonh_wendell, _tx: #ubuntu-classroom-chat please and add "QUESTION:" to your questions 12:06 < giskard> _tx, we have autotools for this kind of stuff (if i understand correctly your question)(btw, question on -chat) 12:06 < nmsa> C(01:06) <@dholbach> mkdir 101; cd 101; wget http://daniel.holba.ch/temp/packaging101.tar.gz 12:07 < nmsa> sorry :) 12:07 <@dholbach> I learned most from looking at existing packages and trying to fix them, so let's take a look at the libsexy package. It's the one in feisty, I broke a bit to illustrate a few common problems you'll face in the daily work as a package maintainer or Ubuntu developer. 12:07 <@dholbach> If you untar the packaging101 tarball, you'll find the following files: 12:07 <@dholbach> libsexy_0.1.10-1ubuntu1.diff.gz libsexy_0.1.10-1ubuntu1.dsc libsexy_0.1.10.orig.tar.gz 12:07 < _tx> Some things I write don't work well with ppc or x86 but work well on the x86_64 namely my system. 12:08 <@dholbach> Together they make the source package. The .orig.tar.gz is called the 'original upstream tarball', the '.diff.gz' file is the changes we need to do to make the package buildable and working correctly in Ubuntu. The .dsc provides meta information like checksums, etc. 12:08 <@dholbach> To unpack the source package run 12:08 <@dholbach> dpkg-source -x libsexy_0.1.10-1ubuntu1.dsc 12:08 <@dholbach> If you change into the directory and have a look into it, you'll find a debian/ directory, which contains the packaging information. 12:09 <@dholbach> Let's try to build the package now. You can use dpkg-buildpackage, but running debuild is easier mostly. So to run a build, simply type 12:09 <@dholbach> debuild 12:09 <@dholbach> 12:09 <@dholbach> in the directory. 12:09 < _tx> Question:How well do users give you feedback here? 12:09 <@dholbach> The friendly build script might now ask you to install some build dependencies, after you did that, it should be spitting out some interesting messages. 12:10 <@dholbach> _tx: in #ubuntu-classroom-chat 12:10 < mattl> jono: all set? 12:10 <@dholbach> in my case the build is complaining (among other things) about the following: 12:11 <@dholbach> sexy-tooltip.c: In function 'sexy_tooltip_new_with_label': 12:11 <@dholbach> sexy-tooltip.c:143: warning: implicit declaration of function 'gtk_label_new' 12:11 <@dholbach> sexy-tooltip.c:143: warning: assignment makes pointer from integer without a cast 12:11 <@dholbach> sexy-tooltip.c:144: warning: implicit declaration of function 'gtk_label_set_markup' 12:11 <@dholbach> sexy-tooltip.c:144: warning: implicit declaration of function 'GTK_LABEL' 12:11 <@dholbach> It's a problem I added myself. :-) 12:11 < jonh_wendell> :) 12:11 <@dholbach> We will need to patch the source to make it build again. 12:11 <@dholbach> If you have a look at debian/rules you will see that it mentions /usr/share/cdbs/1/rules/simple-patchsys.mk - after working with packages for a while that it means you can use cdbs-edit-patch to add a patch. Martin Pitt (pitti) will give a separate session on how to patch packages on Thursday, 18:00 UTC. 12:12 <@dholbach> First we run fakeroot debian/rules clean to clean up our existing build again. 12:12 <@dholbach> next we run cdbs-edit-patch 01-fix-gtk-breakage 12:13 <@dholbach> as I said before, pitti will explain what happens then :-) 12:13 < pitti> correction: patching source packages will be Tuesday 18:00 and Thursday 17:00 UTC 12:13 <@dholbach> thanks pitti 12:14 <@dholbach> (Note: cdbs-edit-patch needs cdbs installed.) 12:14 <@dholbach> Once we're presented with a subshell, in a directory like /tmp/cdbs-new-patch.R29983/libsexy-0.1.10.new , we edit libsexy/sexy-tooltip.c with our editor of choice 12:15 <@dholbach> and add a missing #include 12:15 <@dholbach> just below the existing #include 12:15 <@dholbach> now save the file, and hit Ctrl-D 12:16 <@dholbach> ls debian/patches/ should show our shiny new patch 12:16 <@dholbach> Now we run debuild again and see if we fixed the package. 12:17 <@dholbach> It looks like our attempt succeeded. 12:18 <@dholbach> ls ../*.deb will show us the packages we made 12:18 <@dholbach> I'll answer a bunch of questions after the next step. 12:18 <@dholbach> As I explained above, part of the packaging work is trying to make sure that a package builds in a minimal environment also. Why? 12:19 <@dholbach> Because on our build daemons, a minimal system will be created for each package build to make sure it works in every, even a fresh, scenario. 12:19 <@dholbach> We will now generate a new source package with the changes we made earlier. 12:19 <@dholbach> fakeroot debian/rules clean 12:19 <@dholbach> debuild -S -sa 12:20 <@dholbach> should take us there. 12:20 <@dholbach> cd ..; sudo pbuilder build libsexy_0.1.10-1ubuntu1.dsc should give the pbuilder some work and we can take a look at a few questions. 12:20 < guebay> What is the diff between debuild and dpkg-buildpackage? 12:20 <@dholbach> QUESTION: what's cdbs 12:21 <@dholbach> cdbs is a collection of scripts that make packaging, especially writing debian/rules (a Makefile to automate the package build) a lot easier, by simplifiying common tasks. 12:21 <@dholbach> QUESTION: Will the script disregard temporary editor files when it does the patch ? 12:22 <@dholbach> TLE: I'm not sure what you mean with editor files 12:22 < giskard> xeruno, and.. you can read the package Description: (apt-cache show cdbs) 12:22 < giskard> dholbach, i guess .swp or file~ 12:22 <@dholbach> It's better to get rid of them. They will show up in the .diff.gz and in case of binary files make the build fail. 12:23 <@dholbach> QUESTION: is there a short manual on how to make simple packages? like to create package for krusader from source? 12:23 <@dholbach> I will later post some links to help with packaging, but I fear that packaging will always be a bit tough in the beginning. 12:23 <@dholbach> QUESTIION: Will debuild automagically add all patches in the debian/patches directory 12:24 <@dholbach> amarillion: the rules in debian/rules (namely simple-patchsys.mk) takes care of that 12:24 <@dholbach> QUESTION/ASSERTION: It only succeeds if my name is Daniel Holback 12:24 < giskard> ahah 12:24 <@dholbach> bettsp: I'm sorry, yes you're right. Either disregard the message or run debuild with -k 12:25 <@dholbach> QUESTION: you refer to 'our build daemon' is that kind of pbuilderd ? 12:25 <@seb128> (it works too when it can't sign the package) 12:25 <@seb128> (that's just a warning) 12:25 < bettsp> dholbach: You forgot to run the S-expression I added in the next line to fix my misspelling of your name :) 12:25 <@dholbach> proppy: the build daemons ( https://launchpad.net/+builds ) use sbuild, which is very similar 12:25 <@dholbach> bettsp: yes, I should have said "please add" 12:26 <@dholbach> Ok, my pbuilder is 'finished' now - we'll get back to questions later again. 12:26 <@dholbach> funnily enough the build fails with: 12:26 <@dholbach> checking for PACKAGE... configure: error: Package requirements (pango >= 1.4.0, glib-2.0 >= 2.4.0, gtk+-2.0 >= 2.4.0, libxml-2.0) were not met: 12:26 <@dholbach> No package 'gtk+-2.0' found 12:26 <@dholbach> This is because libgtk2.0-dev is not installed in the minimal environment I was talking about, but how does pbuilder know what to install? 12:27 <@dholbach> if you edit debian/control you will find the line Build-Depends 12:27 <@dholbach> This line indicates the packages that need to be installed to build the package correctly. We'll just add libgtk2.0-dev 12:28 <@dholbach> after that we'll run debuild -S again 12:28 <@dholbach> please note that this time we don't need to run fakeroot debian/rules clean why? 12:29 < proppy> (cause we build using pbuilder) 12:29 <@dholbach> because the build happened in that clean environment, not in the source dir - another good point about using pbuilder 12:29 <@dholbach> let's run sudo pbuilder build libsexy_0.1.10-1ubuntu1.dsc again and see if we get it working now 12:30 <@dholbach> QUESTION: What all does fakeroot debian/rules clean do? 12:30 <@dholbach> levander: it runs the clean target of the upstream source (so whatever the upstream maintainer thought should be cleaned), plus dh_clean and rules you set up yourself 12:31 <@dholbach> QUESTION UPDATE, if I edit the source with emacs, emacs creates a temp file, will the script disregard that when it makes the diff for the patch ? 12:31 <@dholbach> TLE: I'm not sure which file you're referring to. I'm emacs illiterate - sombody else please answer. :-) 12:31 <@dholbach> QUESTION: How would we know from the error message mentioning 'gtk+-2.0' that the package we need is 'libgtk2.0-dev'? Is there a convention for figuring this out? 12:32 < nmsa> j #ubuntu-classroom-chat 12:32 <@dholbach> daxelrod: excellent question. 12:32 < engla> Emacs will create a sexy-tooltip.c~ backup file, and that will be included in the diff if you don't remove it 12:32 < engla> likewise will gedit by default 12:32 <@dholbach> One answer to that is: experience with packaging. But there are also some indicators of where to look: 12:33 < jordi> TLE: if you mean backup files, yes, the "debhelper" system is smartyt enough to not ignore them, but get rid of them when you "clean" the package. 12:34 <@dholbach> In the configure script it failed, because it tried to look up in /usr/lib/pkgconfig 12:34 <@dholbach> so if the build fails for you in pbuilder, but works if you run on your system, you can easily check that directory and find out, where and why it went wrong 12:34 <@dholbach> apt-cache search <...> or apt-file search <...> can help also 12:35 <@dholbach> The pbuilder build worked for me right now, but there are still some things that are weird: 12:36 < giskard> check configure.ac is another solution ;) 12:36 <@dholbach> If you run dpkg -c /var/cache/pbuilder/result/libsexy-doc_*.deb (which displays the contents of the package), you will find it's nearly empty 12:36 <@dholbach> it merely contains a couple of changelogs and copyright notices - what went wrong? 12:37 <@dholbach> If we look into the debian/ directory, you will see that it contains a couple of *.install files 12:37 <@dholbach> these files mention which directories and files are supposed to be installed into which package 12:38 <@dholbach> debian/libsexy-doc.install is empty 12:39 <@dholbach> if you run debuild (to build the package again) and dh_install --list-missing afterwards, it will present you with a list of files that were not installed into any package 12:40 <@dholbach> once we add debian/tmp/usr/share/gtk-doc/html to debian/libsexy-doc.install it will contain something 12:40 <@dholbach> now we can run debuild binary to not have to go through the whole procedure again, but only rebuild the packages 12:41 <@dholbach> You will find that if you run dpkg -c libsexy-doc*.deb again, it will contain the documentation 12:42 <@dholbach> I introduced some more bugs into the package, but given that we only have 19 minutes left, we should probably move on to some more questions. 12:43 <@dholbach> QUESTION: debian/libsexy-doc.install doesn't exist 12:43 <@dholbach> hum, it existed for me - anyway, it was supposed to be an empty file. 12:43 <@dholbach> Q: install pbuilder is still on, slow connection, will it install a new env. will get all packages from net? is doing validation on retrieved packages ... 'thnx 12:44 <@dholbach> nmsa: it's not installing all packages, but only a very minimal set, needed to build packages - cjwatson: which packages are considered by debootstrap? 12:45 <@dholbach> build-essential will be contained in any case. 12:45 <@dholbach> QUESTION: Are *.install files cdbs specific? 12:45 <@dholbach> Florob: no, they're not. They're scanned by dh_install. 12:46 <@dholbach> QUESTION: which is the preferred format for documentation? Should you provide docs in a certain format if they aren't provided upstream? 12:46 <@dholbach> manpages are a good example of documentation that sometimes does not exist. 12:46 <@dholbach> It's always good to push it back to the upstream developers afterwards again. 12:47 <@dholbach> I don't have a link on manpages handy, but I could find you one. I have a template I always use to get started on them. 12:47 < at2000> which packages are considered by debootstrap? the list is in the script /usr/lib/debootstrap/scripts/ 12:47 <@dholbach> QUESTION: is the *.install file method the correct way for one source package to have multiple binary targets? like a lib, lib-dev, -doc and so on? 12:48 <@dholbach> rmjb: some people use *.dirs files or run dh_install to debian/ on their own, I personally prefer the .install file method 12:48 <@dholbach> QUESTION: how about policy on packages. I read executables on /usr/bin must have manpages. 12:48 <@dholbach> manpages are not a strict must have, but they come in very handy and users are gratefule for manpages 12:48 <@dholbach> QUESTION: Is it possible to have pbuilder setup to work in multiple releases concurrently? 12:49 <@dholbach> somerville32: there's a page on the MOTU wiki about that - if nobody finds it out in the next minutes, I can try after the talk 12:49 <@dholbach> QUESTION: can i build for any version deboostrap supports? 12:50 < sol> Should all packages install to /usr? What's the use of /usr/local? 12:50 < sol> Should all packages install to /usr? What's the use of /usr/local? 12:50 <@dholbach> xhaker: you can either set up a chroot for yourself (I think that's what most developers do), because that way you can test the resulting binary packages yourself, or you can have multiple differently configured pbuilders 12:50 < gnomefreak> somerville32: https://wiki.ubuntu.com/PbuilderHowto?highlight=%28pbuilder%29 12:50 <@dholbach> sol: /usr/local is for software you install yourself (like if you use ./configure && make && sudo make install) 12:51 <@dholbach> QUESTION: pbuilder extracts the minimal system and then downloads additional dependencies.. does it cache them? for how long? where? 12:51 <@dholbach> xhaker: yes, it does in /var/cache/pbuilder/aptcache/ 12:51 < xhaker> ;) 12:51 <@dholbach> I think it stores the packages until they're superseded by a newer version 12:52 <@dholbach> are there any other questions? 12:52 < gnomefreak> dholbach: i grabbed the page for somerville32 12:52 <@dholbach> If you run into any other packaging related trouble and don't know where to look in the documentation, your first check could be similar packages, then ask the people on #ubuntu-motu or on ubuntu-motu@lists.ubuntu.com 12:52 < rmjb> thanks for a great session dholbach 12:52 < proppy> dholbach: 18:50 QUESTION(bis): can i chroot inside a pbuilder 12:52 < proppy> environment to debug a packaging failure ? 12:53 < jonh_wendell> do we have to do a clean before every debuild? even if we change just 1 char in some file? 12:53 < engla> thanks, I learnt a lot (even though I built debs before) 12:53 <@dholbach> proppy: yes, you can sudo pbuilder login can help with that 12:53 < proppy> dholbach: thx 12:53 < cjwatson> debuild cleans for you unless you use the -nc option 12:53 < guebay> dholbach: thumbs up :) 12:53 <@jono> ok five min break 12:53 <@jono> till the next session 12:53 < Jucato> yay! 12:54 <@dholbach> jonh_wendell: no you don't - you can run debuild binary to benefit from a previous build and just restart from wherever the build failed 12:54 <@dholbach> thanks everybody 12:54 < amnesia> dholbach: thanks for the session, good job! 12:54 <@dholbach> see you in #ubuntu-motu or on ubuntu-motu@lists.ubuntu.com 12:54 < Florob> jono: great, now it really feels like school ;) 12:54 <@dholbach> thanks a lot 12:54 < sol> thanks :-) 12:54 < fernando> dholbach: thanks 12:54 < amarillion> Thanks dholbach. That is certainly a lot of information to absorb 12:54 < nmsa> dholbach: thank you 12:54 < ifmy> dholbach: thanks 12:54 < scott> no thank you dholbach 12:54 < xhaker> see you in motu 12:54 < DraxNS> nice class dholbach 12:54 < somerville32> Thanks :) 12:55 < amarillion> Florob, not unless you get homework 12:55 < thekorn> thanks dholbach 12:55 <@jono> :P 12:55 < alfmatos> dholbach, nice, thanks a bunch. 12:56 < somerville32> The PBuilderHowto page is kind of confusing. 12:56 <@dholbach> sorry, I forgot: https://wiki.ubuntu.com/MOTU/Documentation <========0 12:57 < berndl> a question which has partly to do with packaging: are there plans on creating a community driven repository (with really up to date packages) like arch-linux has? 12:58 < engla> it would be nice with not only a log, but a cleaned-up summary on the wiki pages (with links). But you know, we could all help out with that if we get to start with the log 12:58 < engla> that way this week could be saved in a box for all 12:58 <@jono> engla, I am going to encourage people to summarise the sessions too 12:58 <@jono> would be useful 12:58 < LaserJock> berndl: that would be the Universe and Multiverse repository 12:58 < siretart> berndl: sort of. I have plans to make REVU, a reviewing platform doing something similar 12:58 < engla> jono: I'm thinking we could link the schedule items to subpages 12:59 <@jono> right 12:59 < engla> I'll help 12:59 <@jono> :) 12:59 < nalioth> engla: jono if you like, you can use w.u.c/ClassroomTranscripts 12:59 <@jono> the sessions so far have been awesome 12:59 <@jono> nalioth, :) 13:00 < berndl> laserjock: yeah, sort of :) but i think it is much easier to get into packaging for everyone in arch than in ubuntu. but perhaps this is the reason we're here to make things easier... :) 13:00 < elvstone> phew. 13:00 < elvstone> ..and sneaks away for some nicotine. 13:00 < berndl> siretart: do you have any specs online for revu? 13:00 <@jono> ok, we all set? 13:00 < Smiffeh> Wonder if Ask Mark tomorrow will end up being productive, or just an excuse for people to quiz him on his openSuSE post. 13:01 < nalioth> probably be a +mz kind of day