[f2py] F2PY: Unable to prevent Fortran ordered array copy
Ravi Kanda
rkanda at gps.caltech.edu
Sat Nov 21 03:49:03 EET 2009
Robin,
I had a "Rip Van Winkle" moment after seeing your post. Your suggestion
worked great - I upgraded Numpy to the latest stable version (1.3), and
the F2PY array-copy problem disappeared. Thanks for the simple, but
valuable tip!
Ravi.
-------------------------------------------------------------------------
Robin wrote:
> On Thu, Nov 19, 2009 at 11:26 PM, Ravi Kanda <rkanda at gps.caltech.edu> wrote:
>> The version of F2PY I am using is identical to that listed as the
>> "2.x/F2PY-2-latest.tar.gz" on the official F2PY page
>> (http://cens.ioc.ee/projects/f2py2e/#download). That corresponds to
>> version F2PY-2.45.241_1926 (dated 30-Jan-2005, in the f2py2e/2.x
>> directory). Is there a more recent stable version that is being
>> maintained elsewhere? I couldn't find F2PY v.2_7618 at the above location.
>>
>> However, I do notice that I am using scipy_distutils-0.3.3_33.572 (also
>> dated 30-Jan-2005) - but there is a newer linux version
>> (scipy_distutils-0.3.3_34.586, dated 19-Jul-2007) listed under
>> "2.x/scipy_distutils-latest.tar.gz"). Perhaps it is worth trying the
>> newer distutils to see if the problem goes away. Do you know if it is
>> possible to upgrade distutils from F2PY without re-installing F2PY? Or
>> do I have to clean install F2PY - i.e., remove F2PY related files from
>> $PYTHONPATH/bin & <PYTHON LIBPATH>/site-packages, then re-install F2PY -
>> and let it detect this newer version of distutils automatically?
>
> All that stuff is really out of date. Everything you need is now
> distributed in numpy (this is where the recent version of f2py lives -
> see news at http://cens.ioc.ee/projects/f2py2e/ ) and scipy_distutils
> is now numpy.distutils - also included in the standard distribution.
> The latest version is 1.3, but 1.4 should be released soon I think.
>
> See:
> http://www.scipy.org/
> http://www.scipy.org/Download
> http://sourceforge.net/projects/numpy/files/
>
> Cheers
>
> Robin
>> Regarding the fortran compiler, I have tested the wrapping with gfortran
>> from gcc 4.2.0, so looks like that is the same version you are using.
>>
>> Thank you for the prompt response,
>> Ravi.
>> --------------------------------------------------------------------------
>>
>> Robin wrote:
>>> Hi,
>>>
>>> Your scripts seemed to work for me with no copies reported. Being on a
>>> Mac I had to disable -DREPORT_AT_EXIT and numpy.testing.memusage() is
>>> unavailable so I didn't get those figures... but the copy warning
>>> works and I didn't get it.
>>>
>>> I am using gfortran 4.2, f2py 2_7618, numpy 1.4.0.dev7618 (mac)
>>>
>>> You are using quite an old version of numpy/f2py, so it would probably
>>> worth upgrading, at least to the latest release to see if the problem
>>> persists.
>>>
>>> Robn
>>>
>>> On Thu, Nov 19, 2009 at 10:01 PM, Ravi Kanda <rkanda at gps.caltech.edu> wrote:
>>>> Thank you, Robin & Labrach, for your comments.
>>>>
>>>> The problem I am experiencing boils down to why the F2PY-module is
>>>> unable to identify a legal NUMPY array to be in the correct order, or
>>>> even recognize legal Numpy array objects! And the problem persists
>>>> irrespective of the F90 compiler I use (Intel, PG, or GNU). The crux of
>>>> this problem may be illustrated with the following two examples from the
>>>> python interpreter ('pymm' is the f2py-wrapped module):
>>>> --------------------------------------------------------
>>>> >>> # Example 1. ROW-MAJOR-ORDERED ARRAY, 'a':
>>>> ...
>>>> >>> a = reshape(arange(15),(3,5))
>>>> >>> a
>>>> array([[ 0, 1, 2, 3, 4],
>>>> [ 5, 6, 7, 8, 9],
>>>> [10, 11, 12, 13, 14]])
>>>> >>> isfortran(a)
>>>> False ---------> OK
>>>> >>> a.flags['F_CONTIGUOUS']
>>>> False ---------> OK
>>>> >>>
>>>> >>> pymm.has_column_major_storage(a)
>>>> 0 ---------> OK
>>>> >>> a = pymm.as_column_major_storage(a)
>>>> Traceback (most recent call last):
>>>> File "<stdin>", line 1, in <module>
>>>> pymm.error: expected array object ---------> (!)
>>>> >>>
>>>> --------------------------------------------------------
>>>> ...
>>>> --------------------------------------------------------
>>>> >>> # Example 2. COLUMN-MAJOR-ORDERED ARRAY, 'b':
>>>> ...
>>>> >>> b = reshape(arange(15),(3,5),order='f')
>>>> >>> b
>>>> array([[ 0, 3, 6, 9, 12],
>>>> [ 1, 4, 7, 10, 13],
>>>> [ 2, 5, 8, 11, 14]])
>>>> >>> isfortran(b)
>>>> True ---------> OK
>>>> >>> b.flags['F_CONTIGUOUS']
>>>> True ---------> OK
>>>> >>> pymm.has_column_major_storage(b)
>>>> 0 ---------> (!)
>>>> >>> c = pymm.as_column_major_storage(b)
>>>> Traceback (most recent call last):
>>>> File "<stdin>", line 1, in <module>
>>>> pymm.error: expected array object ---------> (!)
>>>> >>>
>>>> --------------------------------------------------------
>>>>
>>>> So, 'pymm' does not seem to recognize the legal NUMPY arrays, 'a' OR
>>>> 'b', as valid array objects! As a result, the NUMPY array seems to be
>>>> cast into a 'suitable' form by F2PY resulting in array copying when I
>>>> invoke the function 'pymm.mmfunc.mm' [if compiled with
>>>> 'intent(in,out,overwrite)' OR 'intent(in,out)' OR
>>>> 'intent(in,out,overwrite,c)', and passing the appropriate
>>>> F-/C-CONTIGUOUS arrays]:
>>>> "copied an array using PyArray_CopyFromObject: size=1440000, elsize=4
>>>> copied an array using PyArray_CopyFromObject: size=1440000, elsize=4
>>>> copied an array using PyArray_CopyFromObject: size=1440000, elsize=4"
>>>>
>>>> --------------------------------------------------------
>>>> However, when I compile with 'intent(inout)', OR 'intent(inplace)', OR
>>>> 'intent(inout,c)' with appropriate F-/C-CONTIGUOUS arrays, then I get
>>>> the following error:
>>>> "pymm.error: failed in converting 4th argument `a' of pymm.mmfunc.mm to
>>>> C/Fortran array"
>>>>
>>>> --------------------------------------------------------
>>>> So, I want to know whether:
>>>>
>>>> 1) This could be due to a bug in the F2PY version I am using
>>>> [2.45.241_1926] that was later fixed? Or a problem with my installation
>>>> of F2PY itself.
>>>>
>>>> 2) I am using a wrong flag (or not using a specific flag) during F2PY
>>>> compilation (please see CSH & LOG files in TGZ below)?
>>>>
>>>> 3) I am making some subtle mistake while calling the F2PY-module (please
>>>> see 'CALL_MM.PY' & 'MM.F90' file in TGZ)?
>>>>
>>>> Again, all the relevant text files (7 in all) are available at:
>>>> ftp://ftp.gps.caltech.edu/pub/rkanda/MemoryDeallocTest.tgz
>>>>
>>>> I don't know where else to look to try to resolve this problem. So, it
>>>> will be much appreciated if someone can take a look at the F90, PY, PYF,
>>>> CSH, & LOG files, or actually compile/run the module to see whether I'm
>>>> experiencing a "unique" problem here.
>>>>
>>>> Thank you very much for your time!
>>>> Ravi.
>>>> --------------------------------------------
>>>>
>>>> labrach wrote:
>>>>> It seems ok for me with the followings :
>>>>>
>>>>> (but I don't know how to check if the copy appeared or not, since the
>>>>> -DF2PY_REPORT_ATEXIT returns an error dealing with undefine reference
>>>>> to 'on_exit')
>>>>>
>>>>> I run that on a Windows with mingw 5.1.4
>>>>>
>>>> <SNIP>
>>>>
>>>> _______________________________________________
>>>> f2py-users mailing list
>>>> f2py-users at cens.ioc.ee
>>>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>>>>
>>> _______________________________________________
>>> f2py-users mailing list
>>> f2py-users at cens.ioc.ee
>>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>> --
>> ---------------------------------------------------------------------
>> Ravi Kanda
>> Seismological Laboratory, MC 252-21
>> Division of Geological and Planetary Sciences
>> California Institute of Technology
>>
>> 1200 E. California Blvd., Pasadena, CA 91125
>> Phone: 626-395-6971, Fax: 626-564-0715
>> Web Page: http://www.gps.caltech.edu/~rkanda
>>
>> ----------------------------------------------------------------------
>>
>> For a human being, the unexamined life is not worth living - SOCRATES
>>
>> ----------------------------------------------------------------------
>>
>> _______________________________________________
>> f2py-users mailing list
>> f2py-users at cens.ioc.ee
>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>>
>
> _______________________________________________
> f2py-users mailing list
> f2py-users at cens.ioc.ee
> http://cens.ioc.ee/mailman/listinfo/f2py-users
--
---------------------------------------------------------------------
Ravi Kanda
Seismological Laboratory, MC 252-21
Division of Geological and Planetary Sciences
California Institute of Technology
1200 E. California Blvd., Pasadena, CA 91125
Phone: 626-395-6971, Fax: 626-564-0715
Web Page: http://www.gps.caltech.edu/~rkanda
----------------------------------------------------------------------
For a human being, the unexamined life is not worth living - SOCRATES
----------------------------------------------------------------------
More information about the f2py-users
mailing list