Modifying an SME Server package

Raise a Bugzilla entry

Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug before you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.

All changes must have an associated Bugzilla entry. The bug tracker is here: http://www.contribs.org/bugzilla/

If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:

For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" http://www.contribs.org/bugzilla/show_bug.cgi?id=1174.

Choose the package(s) to modify

If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run rpm -qf on the file to be modified:

[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys 
smeserver-yum-1.1.2-05

and so, we want to modify the smeserver-yum package.

Note: You can view a complete list of the packages checked into SourceForge CVS http://smeserver.cvs.sourceforge.net/smeserver/. On rare occasions, the sources of particular packages may be slightly out of date as there is a small delay (typically only a few hours) between changes to the developer CVS and the anonymous CVS. In practice, this rarely matters and at worst it requires a merging of your changes with any other recent changes to the package.

All packages on the SME Server ISO/CD must be checked into SourceForge CVS. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.

mzgetsf_anon: SourceForge anonymous CVS checkout

You can now retrieve one of the packages from SourceForge. In this case, we want to modify the smeserver-yum package, so let's retrieve it from SourceForge:

[gordonr@smebuild smeserver]$ mzgetsf_anon smeserver-yum
U smeserver-yum/ChangeLog
U smeserver-yum/F/smeserver-yum.spec
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch2
U smeserver-yum/P/smeserver-yum-1.2.0-ModifyUpdateDBs.patch
U smeserver-yum/S/smeserver-yum-1.2.0.tar.gz

Note: Use mzgetsf if you have a SME Server developer account.

Mezzanine package hierarchy

[gordonr@smebuild smeserver]$ cd smeserver-yum/

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S  smeserver-yum

If you examine the checked out package, you will see the following directories:

Note: The smeserver-yum subdirectory is a leftover from an incorrect package import. It can be ignored and will be removed when maintenance is next performed on the CVS repository.

mzclean: Clean out modified files

Before making changes to a package, you should ensure that you have a clean copy of the latest sources. Remember - other people may also be working on the package and making changes in SourceForge. You can perform a "merge" when you check in your changes, but it is easiest to start with an up-to-date copy. It is even possible that someone else has already fixed the bug you are working on!

[gordonr@smebuild smeserver-yum]$ mzclean
Cleaning and resyncing smeserver-yum, please wait....
Removing work....
Cleanup of smeserver-yum complete.

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S

Tip: Ensure that you have copied any modified files out of your work tree prior to runing mzclean.

Note that the stray smeserver-yum subdirectory has been removed.

mzprep: Build working tree

The mzprep command runs rpmbuild -bp on your SPEC file, sources and patches to generate a working tree. You can make changes in the working tree and use mezzanine to generate the patches between the previous version and your new files.

[gordonr@smebuild smeserver-yum]$ mzprep
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
You may now chdir to work to make changes.
Use "mzpatch -n <patch_name>" to generate a patch when done.

[gordonr@smebuild smeserver-yum]$ ls
ChangeLog  CVS  F  P  S  work

The only obvious change is the creation of a work/ subdirectory. Let's have a look in there. Under the work/ directory is smeserver-yum-1.2.0/ which contains the patched sources for this version of smeserver-yum.

[gordonr@smebuild smeserver-yum]$ ls work/
smeserver-yum-1.2.0

Under smeserver-yum-1.2.0/ are the createlinks script, the root hierarchy and a few generated files, explained below.

[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/
createlinks      debugsources.list  smeserver-yum-1.1.2-26-filelist
debugfiles.list  root

SME Server package directory layout

Table 14-1. SME Server package directory layout

Directory/FileDescription
createlinks Builds action, panel and initscript links.
po Package translations, if gettext is used.
root The top-level directory of the package, which will be the / directory of the installed system. So, root/usr/lib/perl5/site_perl becomes /usr/lib/perl5/site_perl on the installed system. Note that the somewhat confusing root/root is the /root directory of the installed system.
update-po Helper script to generate translation binary files, if gettext is used.
debugsources.list debugfiles.listGenerated by RPM build - lists this files which are built with debug flags
smeserver-yum-1.1.2-26-filelistGenerated by RPM build - lists the files to be packaged, and their permissions and ownership

The root directory hierarchy

The work/smeserver-yum-1.2.0/root/ hierarchy contains the files which will be installed on the target machine.

[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/root/
etc  sbin  service  usr  var

Modifying a file

OK - now we can actually make a change. We want to modify the file which will be installed as /etc/e-smith/events/actions/yum-import-keys, so the actual file we need to modify is: work/smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys.

Use your favourite editor (in Unix text mode) to modify the file, adding a comment where the change needs to be made. Here's a diff showing such a comment:

[gordonr@smebuild tmp]$ diff -u yum-import-keys.orig yum-import-keys
--- yum-import-keys.orig        2006-05-22 16:28:51.734609534 +1000
+++ yum-import-keys     2006-05-22 16:28:39.006814950 +1000
@@ -30,6 +30,7 @@
 for my $key ( grep {!/^\./} readdir(DIR) )
 {
     warn "Importing key $key\n";
+    # TODO: Skip keys which have previously been imported

     system("rpm", "--import", $key) == 0 or
        warn "Couldn't rpm --import $key\n";

mzpatch: Create a patch

Mezzanine automates the creation of patches through the mzpatch command.

Note: Always run mzpatch from the top-level directory of your package (e.g. ~/smeserver/smeserver-yum/).

[gordonr@smebuild actions]$ cd ~/smeserver/smeserver-yum/

[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ModifyUpdateDBs.patch

You will see from the listing above that the patches should be named package-version-patchname.patch. You should also ensure that the patch name you are about to choose doesn't already exist, or you may overwrite an older version of the patch.

[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
cvs [server aborted]: "add" requires write access to the repository
You do not have write access to the master repository.
srctool:  Error:  Addition of P/smeserver-yum-1.2.0-ImportKeysComment.patch failed.
[gordonr@smebuild smeserver-yum]$ ls P
CVS
smeserver-yum-1.2.0-DisplayStatus.patch
smeserver-yum-1.2.0-DisplayStatus.patch2
smeserver-yum-1.2.0-ImportKeysComment.patch
smeserver-yum-1.2.0-ModifyUpdateDBs.patch

Mezzanine has created the patch, but it has not committed it to CVS. This allows you to revise the patch until it works, prior to committing it. If you do not have a developer account you will get an error when mezzanine attempts to reserve the patch name in CVS, but the patch will still be created in the P directory.

The patch looks quite similar to the one we saw previously, except for the pathname to the modified file:

[gordonr@smebuild smeserver-yum]$ cat P/smeserver-yum-1.2.0-ImportKeysComment.patch
diff -Nur -x '*.orig' -x '*.rej' smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys
--- smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2005-09-26 12:30:54.000000000 +1000
+++ mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys       2006-05-22 16:35:57.752811633 +1000
@@ -30,6 +30,7 @@
 for my $key ( grep {!/^\./} readdir(DIR) )
 {
     warn "Importing key $key\n";
+    # TODO: Skip keys which have previously been imported

     system("rpm", "--import", $key) == 0 or
        warn "Couldn't rpm --import $key\n";

Building a new package, with the patch

The next step is to change the package SPEC file so that the patch is applied.

Update the Release tag

The first change you should make is to update the "Release" tag in the SPEC file. You should do this by appending your initials and sequence number to the Release number. In most SME Server SPEC files, the Release number is set via the release macro, and so you should set it there.

Summary: YUM, an rpm updater
%define name smeserver-yum
Name: %{name}
%define version 1.2.0
- %define release 05
+ %define release 05ff01

Please just append your initials and a sequence number to the release tag. Do not change the package name (e.g. to smeserver-yum-fred) or the version number. This policy makes it obvious that your update is a patch based on smeserver-yum-1.2.0-05.

If a change has been approved by the Development Manager, it is given an official version/release number (e.g. smeserver-yum-1.2.0-06). Always tag unofficial changes with your initials and sequence number.

If an upstream (e.g. CentOS) package needs to be patched and it is not possible to wait for the patch to come from the upstream source, the release number should be modified with the special tag sme and a sequence number (e.g. clamav-0.88.2-1sme01). The sme tag is reserved for changes approved by the Development Manager.

Add a changelog entry

The SME Server project maintains history of packages changes in the %changelog section of the SPEC file. The changelog entry should be a simple description of the change of behaviour and must include a reference to an SME Server project Bugzilla entry (1174) and the new version number (1.2.0-05ff01):

%changelog
+ * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
+ - Initial work on avoiding duplicate RPM keys [SME: 1174]

* Mon May  1 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
- Remove stray yum.pm.orig file. [SME: 1350]

The first line of the changelog entry shows when the change was made, by whom and notes the new version number. The other lines describe the change and refer to the Bugzilla entry or entries, using the format [SME: 1174] as a shorthand.

Add a patch reference

You now need to tell RPM to retrieve and apply the new patch. This needs to be done in two places, which is annoying, but allows for patches to be carried around, but not applied (e.g. because they are not quite finished). The first change is near the top of the file:

Patch0: smeserver-yum-1.2.0-DisplayStatus.patch
Patch1: smeserver-yum-1.2.0-ModifyUpdateDBs.patch
Patch2: smeserver-yum-1.2.0-DisplayStatus.patch2
+ Patch3: smeserver-yum-1.2.0-ImportKeysComment.patch

The second change is in the %prep section:

%prep
%setup
%patch0 -p1
%patch1 -p1
%patch2 -p1
rm root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/yum.pm.orig
+ %patch3 -p1

mzbuild: Build a new package

With all that done, you should be able to build a new package using mzbuild:

[gordonr@smebuild smeserver-yum]$ mzbuild

[...]
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35037
+ umask 022
+ cd /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/build.mezz/BUILD
+ cd smeserver-yum-1.2.0
+ /bin/rm -rf /var/tmp/mezzanine-buildroot.18508
+ exit 0
Package build succeeded.  Output files are:

smeserver-yum-1.2.0-05ff01.src.rpm
smeserver-yum-1.2.0-05ff01.noarch.rpm

Check the package

You should now perform some simple checks on the package before installing it for testing.

  • Does the built package have your initials in the release tag? We never want to have two packages with the same name and version number.

    [gordonr@smebuild smeserver-yum]$ ls *.rpm
    smeserver-yum-1.2.0-05ff01.noarch.rpm  smeserver-yum-1.2.0-05ff01.src.rpm
      
  • Did you update the changelog? It's easy to forget, but vital to do.

    [gordonr@smebuild smeserver-yum]$ rpm -qp --changelog \
      smeserver-yum-1.2.0-05ff01.noarch.rpm | head -8
    
    * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
    
    - Initial work on avoiding duplicate RPM keys [SME: 1174]
    
    * Mon May 01 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
    
    - Remove stray yum.pm.orig file. [SME: 1350]
      
  • Did you remember to apply the patch? It happens, and leads to head scratching "I knew I made that change".

    [gordonr@smebuild smeserver-yum]$ mzbuild | grep patch
    + echo 'Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):'
    Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):
    + patch -p1 -s
    + echo 'Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):'
    Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):
    + patch -p1 -s
    + echo 'Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):'
    Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):
    + patch -p1 -s
    + echo 'Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):'
    Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):
    + patch -p1 -s
      

Test your new package

You should now upgrade your test system:

[gordonr@sevendev1 tmp]$ sudo rpm -Uvh smeserver-yum-1.2.0-05ff01.noarch.rpm
Preparing...                ########################################### [100%]
   1:smeserver-yum          ########################################### [100%]

and test that you have fixed the bug.

Lather, rinse, repeat

If you haven't fixed the bug, repeat the cycle: make changes, build a new patch, update the release number (e.g. to 05ff02, 05ff03, etc.), build a new package, test. Don't worry about using up release numbers.

Note: Actually fixing bug 1174 is left as an exercise for the reader. Fame and fortune (or at least thanks) await the person who does.

Attach your patch(es) to the bug

You should now attach your patch(es) and SPEC file changes to the appropriate Bugzilla entry. If the change is the simple application of a patch, you do not need to attach the whole SPEC file - just the changelog entry.

Please attach the patch(es) and ensure that they are in Unix text format. Patches make it obvious which sections of code have been changed and attaching them ensures that their formatting is preserved.