MantisBT

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001415MPDOtherpublic2006-12-14 07:482009-01-04 16:03
Reporterlostclus 
Assigned Tometyl 
PrioritylowSeverityfeatureReproducibilityN/A
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Versiongit 
Summary0001415: [PATCH] pkzip, gzip and bzip2 on-fly decompression
DescriptionThis patch adds pkzip, gzip and bzip2 decompression feature in mod_plugin.c. Also fixes module infinitive loops.
Tagsarchive, zip
Attached Filespatch file icon mpd-0.12.1-unpack.patch [^] (25,433 bytes) 2006-12-14 07:48 [Show Content]
gz file icon mpd-archiveapi-preview.tar.gz [^] (46,972 bytes) 2008-11-23 10:30
gz file icon mpd-archiveapi-v1.tar.gz [^] (55,506 bytes) 2008-11-27 20:59
gz file icon mpd-archiveapi-v2.tar.gz [^] (64,602 bytes) 2008-12-07 16:13

- Relationships
has duplicate 0000454closedAvuton Olrich Adding ZIP-Support for mod-files 
has duplicate 0000566closedcirrus Adding new inputStream into mpd 

-  Notes
(0001138)
Avuton Olrich (administrator)
2006-12-19 10:38

lostclus: I'd like to see this patch make it in. Would you please split this patch into two patches, one for each cause? It will probably take a little longer to get the decompression stuff in, but I'd like to get the fix for module infinite loops in now, if possible. Thanks.
(0001143)
Avuton Olrich (administrator)
2006-12-19 11:18

lostclus: Also, Tried adding a mod zip directly from modarchive.org, which didn't initially add to the database, I guess it requires recursive unarchiving. Looks like:

TMA2006-snapshot-1_9.zip

When I ran an update didn't work. When I unzipped it I saw:

1_9/106.xm.zip
...

It did work after this update. Any chance on extending this patch to get something like this working? Thanks again.
(0001144)
lostclus (reporter)
2006-12-19 14:30

Avuton Olrich: please give me a link to the file on modarchive.org that require recursive unarchiving.
(0001145)
lostclus (reporter)
2006-12-19 14:42

Sorry. I have found it in http://tracker.modarchive.org/. [^] So, this is snapshot of entire directory, that contains multiple files. Archives with multiple files is not supported by patch. Only first file will be extracted from multifile zip archive and this file MUST be in one of formats that libmikmod can recognize.
(0001146)
Avuton Olrich (administrator)
2006-12-21 13:29

You didn't respond to the request to split out the patch to

1) fix
2) features

Secondly, that means this patch has /very/ limited audience, to be honest seems like a ton of trouble for no gain, that doesn't mean I'm totally opposed but I'm not left with any reason to go and support it. I just bzip -8 >1000 mods and they hardly compressed in the first place. On the other hand, if it was more general where it would 'find' things such as mp3s in zips, or where I could zip 18,000 mods in a zip (which would keep my disk from allocating alot more space for it then it needs). I just need to understand why I'm going upto bat for a 902 line patch that has very limited functionality, perhaps I'm missing the point? Thanks.
(0001151)
lostclus (reporter)
2006-12-21 14:16

> You didn't respond to the request to split out the patch to
> 1) fix
> 2) features

First, there is http://www.musicpd.org/mantis/view.php?id=1392 [^] but it still
without any attention. Second, I posted in 2004
http://lostclus.linux.kiev.ua/patches/all/mpd-SVN20041115-nomodloops.patch [^] .
Ignored.

> On the other hand, if it was more general where it would 'find' things such
> as mp3s in zips, or where I could zip 18,000 mods in a zip (which would keep
> my disk from allocating alot more space for it then it needs). I just need to
> understand why I'm going upto bat for a 902 line patch that has very limited
> functionality

http://lostclus.linux.kiev.ua/patches/all/mpd-SVN20041115-bzip2.patch [^] .
Rejected in 2004.
(0001153)
Avuton Olrich (administrator)
2006-12-21 15:03

I understand things have gotten ignored in the past. We get busy, things get pushed aside. But everything aside, I'm *trying* to get things done here. I *will* do everything in my power to get this into the trunk. I do have some qualms at the moment though.

I can get the fix imported near-immediately. I can get the feature queued for review, but I need you to work with me here. I can probably even do this stuff myself, but with my limited coding ability it maybe bug worthy and take even longer. I *will* get this into testing at least. I would like to host this on my personal MPD branch. I *have* been testing this for a few days now, myself. What I do need at this point is the attached patch split in

1) Fix
2) Feature

It's going to make it easier to review, easier to sync, and it's really appreciated. If you don't get around to it I may or may not.

So here's the list of things that I need or need to understand.
1) Patch split in two parts.
2) I also see that if --disable-mod configured, bzip2/zip support still gets compiled in. It's be great to see that configured out if mod is disabled, until/unless the compression patch becomes more powerful.
3) It would be *nice* to see a patch from the latest svn.

Please understand this is all appreciated, I will work on getting this included. Thanks!
(0001154)
Avuton Olrich (administrator)
2006-12-21 15:17

Is the patch in 1392 a complete fix for the looping problems, lostclus? Thanks.
(0001170)
lostclus (reporter)
2006-12-22 22:35

English is not my native language, so that I will not speak about all things
that I have. Whether or not this changes will be accepted is all the same for
me. I give you the idea, moreover I give you the code, that even works. Rest is
your matter. If you want, you can simply include this changes as is. If you
want, you can modify it to fit your desires. If you want, you can reject it. I
consider, if you can read and understand program code, then you can spit this
patch into so much pieces as you like without any difficulties. If you can't
(don't want) to read and understand the program code, then we can't understand
one another.
(0002420)
metyl (reporter)
2008-11-19 14:14
edited on: 2008-11-27 23:21

in my opinon on compression in mpd. compression containerts should be implemented on the inputfile_* / directory level. so that every gz/bz2/zip archive can be transparently scanned for containing files and any decoder then can use the corresponsing feature of taring album files together.

There should be some un/archive api proposal on which we can add future compression methods. I would like to see .zip and .rar support along with unarchive api for mpd proposal. I would like to cooperate on this thema.

from mpd browser point of view an archive should look like directory, for. ex
> On the other hand, if it was more general where it would 'find' things such
> as mp3s in zips, or where I could zip 18,000 mods in a zip (which would keep
> my disk from allocating alot more space for it then it needs). I just need to
> understand why I'm going upto bat for a 902 line patch that has very limited
> functionality
/musicroot/Talco - Combat Circus.zip/Talco/11 - A LA Pachenka.mp3

Free music for free people
http://www.jamendo.com/en/track/39231 [^]

(0002421)
cirrus (administrator)
2008-11-19 14:27

metyl: agree. A good API which abstracts this stuff would be great, and could provide an input_stream implementation, as well as enumeration (for database update). Nobody is currently working on that - if you have ideas, please share them - the IRC channel or the developer mailing list would be the best medium for discussion.
(0002424)
metyl (reporter)
2008-11-19 21:56
edited on: 2008-11-19 21:58

i have written some stuff today, trying to put api together. the main issues are listed here:

1.) archive plugins needs to be stackable. For example on top of gzip/bzip2 plugin can sit tar plugin which lists / reads the files. While this looks easy it also adds a new level of problems (when seeking is imposible, tar plugin needs to read and process all data in stream as it goes).

there could be problem implementing this to work transparently. Imagine, reading files from archive randomly can significally raise load in mpd because whole tar file needs to be decoded (for tar.gz tar.bz2 files)

2.) it looks like compression has no sence at all because music files are generally compressed better. only grouping of multiple files is wanted.

because of this i see sence for implementing support for
.zip .rar .tar containers only when compression is disabled

i made some initial coding without really thinking how big issue this is
going into death end. If anybody interested i'll post patch to git

there is a lot of things todo here

(0002427)
cirrus (administrator)
2008-11-20 12:53

Just a thought: might gnome-vfs be appropriate for us? I don't know it, but maybe it's worth to check.

We don't need to support tarfiles. Like you said, the overhead is huge, since they have no central file listing.

Even supporting compressed zip files could be dropped for simplicity now.
(0002430)
metyl (reporter)
2008-11-20 19:47
edited on: 2008-11-22 17:12

i'm working currently on simplified design. As for first plugin i'm using unrarlib which supports rar only format v 2.0 (newest winrar is using v3). I've droped tar support and plugin stacking to make it easier and less intrusive to get into master branch.

plan is to make support for zip (via zlib) files possibly any other true archive (with separate extract possibility)

\i'm not familiar with gnome-vfs but it will add too much dependency i think

(0002457)
metyl (reporter)
2008-11-23 10:34
edited on: 2008-11-27 23:29

-> first preview. Only scanning of archive files (zip, rar, bz2) is working as now. Update of mpd database still incomplete. This is more like a backup until i get more familiar with git

-> first working version. Tag reading is not supported. but playing from .zip and .bz2 (not tarred only bzip2'ed) is fully functional... rar support is implemented but only for v rar v 2.0 format and rar plugin is not tested at all (I cant get old rar packer). For the mod files support, mod plugin needs to implement stream_decode instead of file_decode to finally allow grouping of mod files....

(0002600)
metyl (reporter)
2008-12-07 16:14

-> second working version: some fixes in directory update code, new iso reading plugin
(0002696)
Avuton Olrich (administrator)
2008-12-28 08:01

Fixed in git, thanks for the contribution.

- Issue History
Date Modified Username Field Change
2006-12-14 07:48 lostclus New Issue
2006-12-14 07:48 lostclus Status new => assigned
2006-12-14 07:48 lostclus Assigned To => shank
2006-12-14 07:48 lostclus File Added: mpd-0.12.1-unpack.patch
2006-12-19 10:36 Avuton Olrich Assigned To shank => Avuton Olrich
2006-12-19 10:38 Avuton Olrich Note Added: 0001138
2006-12-19 10:38 Avuton Olrich Status assigned => feedback
2006-12-19 10:59 Avuton Olrich Relationship added has duplicate 0000454
2006-12-19 11:18 Avuton Olrich Note Added: 0001143
2006-12-19 14:30 lostclus Note Added: 0001144
2006-12-19 14:42 lostclus Note Added: 0001145
2006-12-21 13:29 Avuton Olrich Note Added: 0001146
2006-12-21 14:16 lostclus Note Added: 0001151
2006-12-21 15:03 Avuton Olrich Note Added: 0001153
2006-12-21 15:17 Avuton Olrich Note Added: 0001154
2006-12-22 22:35 lostclus Note Added: 0001170
2006-12-28 23:28 Avuton Olrich Status feedback => assigned
2006-12-28 23:28 Avuton Olrich Assigned To Avuton Olrich => shank
2008-10-25 19:04 Avuton Olrich Assigned To shank => cirrus
2008-10-25 21:57 Avuton Olrich Category => Other
2008-10-27 11:12 cirrus Priority normal => low
2008-11-19 14:14 metyl Note Added: 0002420
2008-11-19 14:18 metyl Note Edited: 0002420
2008-11-19 14:27 cirrus Note Added: 0002421
2008-11-19 21:56 metyl Note Added: 0002424
2008-11-19 21:57 metyl Note Edited: 0002424
2008-11-19 21:57 metyl Note Edited: 0002424
2008-11-19 21:58 metyl Note Edited: 0002424
2008-11-19 21:58 metyl Note Edited: 0002424
2008-11-20 12:53 cirrus Note Added: 0002427
2008-11-20 12:54 cirrus Assigned To cirrus => metyl
2008-11-20 19:47 metyl Note Added: 0002430
2008-11-20 19:48 metyl Note Edited: 0002430
2008-11-22 17:03 metyl Note Edited: 0002420
2008-11-22 17:12 metyl Note Edited: 0002430
2008-11-23 01:19 Avuton Olrich Relationship added has duplicate 0000566
2008-11-23 10:30 metyl File Added: mpd-archiveapi-preview.tar.gz
2008-11-23 10:34 metyl Note Added: 0002457
2008-11-27 20:59 metyl File Added: mpd-archiveapi-v1.tar.gz
2008-11-27 23:21 metyl Note Edited: 0002420
2008-11-27 23:26 metyl Note Edited: 0002457
2008-11-27 23:27 metyl Note Edited: 0002457
2008-11-27 23:29 metyl Note Edited: 0002457
2008-12-07 16:13 metyl File Added: mpd-archiveapi-v2.tar.gz
2008-12-07 16:14 metyl Note Added: 0002600
2008-12-07 16:14 metyl Tag Attached: archive
2008-12-07 16:15 metyl Tag Attached: zip
2008-12-28 08:01 Avuton Olrich Note Added: 0002696
2008-12-28 08:01 Avuton Olrich Status assigned => resolved
2008-12-28 08:01 Avuton Olrich Fixed in Version => git
2008-12-28 08:01 Avuton Olrich Resolution open => fixed
2009-01-04 16:03 Avuton Olrich Status resolved => closed


Copyright © 2000 - 2014 MantisBT Team
Powered by Mantis Bugtracker