OK... now I'm trying to fix the bug but I can't reproduce it. I made a 5 GB soundfile (wave format, 60 channels, 4-bit floats, 450 seconds long) and opened and am reading it using readsf~. This is on a 64 bit linux box.
Question for Charles Goyard: were you on a 32-bit linux machine? Or did you do something else besides just read the file from its beginning?
thanks Miller
On Wed, Mar 20, 2013 at 06:20:59PM -0700, Miller Puckette wrote:
I believe not - every extern that used open_bia_path0 would have to be fixed at the source level to use lseek64(), fstat64() in place of lseek() and fstat(). I can't see any way to "fix" this source-compatibly.
I checked open64() on 32-bit Raspbian (ARM) and it worked fine - so I think it's save to say open64 exists on all modern linuxes, both 64 and 32 bit.
cheers Miller On Wed, Mar 20, 2013 at 09:14:42PM -0400, Ivica Ico Bukvic wrote:
Miller,
Does this mean if one were not so concerned with backwards compatibility and included define you listed below in the s_path.c that this would fix the LFS issue albeit at the expense of backwards/cross-platform compatibility? Also, in Linux is open64 safe for both 32-bit and 64-bit OS variants?
-----Original Message----- From: pd-list-bounces@iem.at [mailto:pd-list-bounces@iem.at] On Behalf Of Miller Puckette Sent: Wednesday, March 20, 2013 6:46 PM To: IOhannes m zmölnig Cc: pd-list@iem.at Subject: Re: [PD] Large File Support on Linux
OK.. I think I'm sold (for 0.45 :) on trying to get sys_open, etc., to do the "right" thing. I guess on Windows this would have to be somehow both UTF8 and 64-bit safe - all the more reason to do as you suggest and hide
the
whole mes behind sys_this_and_that() variants.
I thought exactly the same thing about a find_via_path routine. It seems
to
me that open_via_path, which tries not just to verify that the file exists
but
also that it can actually be opened, is perhaps working too hard; if a
file
that proves impossible to open is in an earlier spot on the path, the best thing might be to try and fail to open it instead of going out to another, perhaps incorrect, version of the file.
Also, it could be fixed to take the path as an argument so that
d_soundfile.c
can call it from other threads safely (using separate copies of the path).
cheers Miller
On Wed, Mar 20, 2013 at 09:42:42PM +0100, IOhannes m zmölnig wrote:
On 03/20/2013 18:02, Miller Puckette wrote:
On further thought, I don't think it's even possible to change
open_via_path()
to use open64() and maintain even source compatibility for externs -
if
one of them calls lseek or fstat we're sunk - and I don't see any
robust
way of aliasing those calls to lseek64() etc.
yep. that's why i said that the only real solution would be to provide a more-or-less complete set of fileIO-functions: sys_lseek, sys_stat (including f-variants).
I'm now realizing too that I don't know what approaches the Mac and
Microsoft
pltforms are taking to large file support - I think any solution will
have to
somehow address all of the platforms.
which would be possible if we provided the full set. of file accessors. (but to repeat, this is not really a Pd-problem, and could (and maybe should) be solved in an auxiliary lib).
For right now I'd like a way to fix 0.44's sfread~ - I'm thinking I
can do
that internally to d_soundfile.c so as to cause the least possible
disruption
in a 'bugfix' pd release (probably 0.44-3).
which i think is a good enough approach. but of course there's a catch: sys_open() will gracefully handle UTF-8 filenames, whereas on some platforms open() will not. so with your solution, soundfiles with non-ascii chars will fail to open on W32.
since open_via_path() is so often used to determine the full path of a file somewhere in the searchpath, it might be a good idea to wrap that functionality into a find_via_path() function that returns the canonicalized filename. it would be great if that filename would be mangled in a way so UTF8 doesn't make problems any more, but i think this is simply not possible with the way w32 handles opening such
filenames.
fgmadsr IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list