Re: yum-deltarpm Was Thread Hijack - Our package management GUI tools need improvement

["Ahmed Kamal"]

The idea of using metadata to conclude whether/not drpm reconstruction will be successful, is possible. This metadata is called a sequence number "SEQ" for some reason. However, at this time, I have not integrated code for that. The current code blindly tries downloading and fails with a 404. Integrating metadata is listed on the TODO on presto page, and it shouldn't be too difficult anyway. However, since drpms are usually too small compared to their comparative rpm, and since in almost all cases on disk files are not going to be corrupt, I did not see much value in using this kind of metadata.

The drpm generator script does keep drpms on the server only if they are worth keeping. The worthfulness numbers, of course can be tuned later. I even think it might be a good idea to make worthfulness depend linearly on the new rpm size. 
i.e. keep drpms for large rpms, even if savings are not that great. We are however getting very good savings on large packages anyway

My main focus now is on testing and making sure the "base" system is working as it should. Any ideas about that regression system? Do you think it's a good idea? I'm not primarily into coding, so I'll need help making sound decisions. I'm thinking of having a full FC6 install, then using drpms to upgrade that into *-testing, that should give us some nice reports for how many upgrades/reconstructions are failing. We'll probably need some server to host the drpms on, plus the test client.

On 3/9/07, Warren Togami <wtogami@xxxxxxxxxx> wrote:

Jonathan Dieter wrote:
> On Fri, 2007-03-09 at 12:35 -0500, Warren Togami wrote:
>> 2) Ahmed Kamal has been working on a potentially sane implementation of
>> deltarpm for Fedora's yum.  Theoretically, it would work as an optional
>> yum plugin.  If the deltarpm is substantially smaller than an RPM
>> update, then the deltarpm is provided on a mirror.  If the deltarpm is
>> not provided, then yum downloads the original RPM instead.  If it
>> downloaded a deltarpm, it reconstructs the original RPM and uses GPG to
>> verify integrity just like yum would verify plain RPM downloads.
>>
>> Ahmed probably could use some developer and testing help.  I've been
>> encouraging him to be more communicative about his project in order to
>> get more help, but I haven't seen any further outreach lately.
>>
>> Warren Togami
>> wtogami@xxxxxxxxxx
>>
> I've been working with Ahmed and Michael Schroeder (the upstream
> maintainer of deltarpm) to track down some long-standing bugs in
> deltarpm, especially as it relates to prelinked binaries.  These bugs
> were causing very odd problems while working with the yum-deltarpm
> plugin.

Interesting!  I had not considered prelinking.  I'm glad you found this
problem.

Does it check the integrity of all other files (except %config) before
deciding to attempt to download the deltarpm?  (If some other file is
modified, abort and download the regular RPM?)

>
> We *think* we've found them all (the patches are in the latest Rawhide
> version of deltarpm in Extras), so I think we're at the point where what
> we really need is someone who would be willing to create drpms of all
> new packages in Core and Extras (there's a modified version of prunerepo
> that does all the work for you), and host them for us.
>
> To give you an idea of the savings we're looking at:
> * kdebase-3.5.5-0.4.fc6  => kdebase-3.5.6-0.1.fc6 =  3.5MB vs. 30.2MB
> * kdegames-3.5.5-0.1.fc6 => kdegames-3.5.6-0.1.fc6 = 740KB vs. 11.1MB
> * OO.o-core-2.0.4-5.5.3  => OO.o-core-2.0.4-5.5.10 = 8.7MB vs. 92.4 MB
> There won't be that kind of savings for all the packages, but a general
> rule of the thumb is that the bigger the package, the better the chance
> that we'll get a good compression ratio on the drpm.

Generated drpms are only to be kept on mirrors if the download savings
are substantial.  Some threshold level has to be decided.

For example (actual numbers can be decided later):
Only push a drpm to mirrors if the download savings are greater than 40%
total size, and greater than 5MB.

Warren Togami
wtogami@xxxxxxxxxx

--
fedora-devel-list mailing list
fedora-devel-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-devel-list

-- 
fedora-devel-list mailing list
fedora-devel-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-devel-list