[f2py] F2PY: Unable to prevent Fortran ordered array copy

Ravi Kanda rkanda at gps.caltech.edu
Sat Nov 21 03:49:03 EET 2009


Robin,

I had a "Rip Van Winkle" moment after seeing your post.  Your suggestion 
worked great - I upgraded Numpy to the latest stable version (1.3), and 
the F2PY array-copy problem disappeared.  Thanks for the simple, but 
valuable tip!

Ravi.
-------------------------------------------------------------------------

Robin wrote:
> On Thu, Nov 19, 2009 at 11:26 PM, Ravi Kanda <rkanda at gps.caltech.edu> wrote:
>> The version of F2PY I am using is identical to that listed as the
>> "2.x/F2PY-2-latest.tar.gz" on the official F2PY page
>> (http://cens.ioc.ee/projects/f2py2e/#download). That corresponds to
>> version F2PY-2.45.241_1926 (dated 30-Jan-2005, in the f2py2e/2.x
>> directory). Is there a more recent stable version that is being
>> maintained elsewhere?  I couldn't find F2PY v.2_7618 at the above location.
>>
>> However, I do notice that I am using scipy_distutils-0.3.3_33.572 (also
>> dated 30-Jan-2005) - but there is a newer linux version
>> (scipy_distutils-0.3.3_34.586, dated 19-Jul-2007) listed under
>> "2.x/scipy_distutils-latest.tar.gz").  Perhaps it is worth trying the
>> newer distutils to see if the problem goes away.  Do you know if it is
>> possible to upgrade distutils from F2PY without re-installing F2PY?  Or
>> do I have to clean install F2PY - i.e., remove F2PY related files from
>> $PYTHONPATH/bin & <PYTHON LIBPATH>/site-packages, then re-install F2PY -
>>  and let it detect this newer version of distutils automatically?
> 
> All that stuff is really out of date. Everything you need is now
> distributed in numpy (this is where the recent version of f2py lives -
> see news at http://cens.ioc.ee/projects/f2py2e/ ) and scipy_distutils
> is now numpy.distutils - also included in the standard distribution.
> The latest version is 1.3, but 1.4 should be released soon I think.
> 
> See:
> http://www.scipy.org/
> http://www.scipy.org/Download
> http://sourceforge.net/projects/numpy/files/
> 
> Cheers
> 
> Robin
>> Regarding the fortran compiler, I have tested the wrapping with gfortran
>>  from gcc 4.2.0, so looks like that is the same version you are using.
>>
>> Thank you for the prompt response,
>> Ravi.
>> --------------------------------------------------------------------------
>>
>> Robin wrote:
>>> Hi,
>>>
>>> Your scripts seemed to work for me with no copies reported. Being on a
>>> Mac I had to disable -DREPORT_AT_EXIT and numpy.testing.memusage() is
>>> unavailable so I didn't get those figures... but the copy warning
>>> works and I didn't get it.
>>>
>>> I am using gfortran 4.2, f2py 2_7618, numpy 1.4.0.dev7618 (mac)
>>>
>>> You are using quite an old version of numpy/f2py, so it would probably
>>> worth upgrading, at least to the latest release to see if the problem
>>> persists.
>>>
>>> Robn
>>>
>>> On Thu, Nov 19, 2009 at 10:01 PM, Ravi Kanda <rkanda at gps.caltech.edu> wrote:
>>>> Thank you, Robin & Labrach, for your comments.
>>>>
>>>> The problem I am experiencing boils down to why the F2PY-module is
>>>> unable to identify a legal NUMPY array to be in the correct order, or
>>>> even recognize legal Numpy array objects!  And the problem persists
>>>> irrespective of the F90 compiler I use (Intel, PG, or GNU).  The crux of
>>>> this problem may be illustrated with the following two examples from the
>>>> python interpreter ('pymm' is the f2py-wrapped module):
>>>> --------------------------------------------------------
>>>>  >>> # Example 1. ROW-MAJOR-ORDERED ARRAY, 'a':
>>>> ...
>>>>  >>> a = reshape(arange(15),(3,5))
>>>>  >>> a
>>>> array([[ 0,  1,  2,  3,  4],
>>>>          [ 5,  6,  7,  8,  9],
>>>>          [10, 11, 12, 13, 14]])
>>>>  >>> isfortran(a)
>>>> False                                   --------->  OK
>>>>  >>> a.flags['F_CONTIGUOUS']
>>>> False                                   --------->  OK
>>>>  >>>
>>>>  >>> pymm.has_column_major_storage(a)
>>>> 0                                       --------->  OK
>>>>  >>> a = pymm.as_column_major_storage(a)
>>>> Traceback (most recent call last):
>>>>     File "<stdin>", line 1, in <module>
>>>> pymm.error: expected array object       --------->  (!)
>>>>  >>>
>>>> --------------------------------------------------------
>>>> ...
>>>> --------------------------------------------------------
>>>>  >>> # Example 2. COLUMN-MAJOR-ORDERED ARRAY, 'b':
>>>> ...
>>>>  >>> b = reshape(arange(15),(3,5),order='f')
>>>>  >>> b
>>>> array([[ 0,  3,  6,  9, 12],
>>>>        [ 1,  4,  7, 10, 13],
>>>>        [ 2,  5,  8, 11, 14]])
>>>>  >>> isfortran(b)
>>>> True                                    --------->  OK
>>>>  >>> b.flags['F_CONTIGUOUS']
>>>> True                                    --------->  OK
>>>>  >>> pymm.has_column_major_storage(b)
>>>> 0                                       --------->  (!)
>>>>  >>> c = pymm.as_column_major_storage(b)
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>> pymm.error: expected array object       --------->  (!)
>>>>  >>>
>>>> --------------------------------------------------------
>>>>
>>>> So, 'pymm' does not seem to recognize the legal NUMPY arrays, 'a' OR
>>>> 'b', as valid array objects!  As a result, the NUMPY array seems to be
>>>> cast into a 'suitable' form by F2PY resulting in array copying when I
>>>> invoke the function 'pymm.mmfunc.mm' [if compiled with
>>>> 'intent(in,out,overwrite)' OR 'intent(in,out)' OR
>>>> 'intent(in,out,overwrite,c)', and passing the appropriate
>>>> F-/C-CONTIGUOUS arrays]:
>>>> "copied an array using PyArray_CopyFromObject: size=1440000, elsize=4
>>>>  copied an array using PyArray_CopyFromObject: size=1440000, elsize=4
>>>>  copied an array using PyArray_CopyFromObject: size=1440000, elsize=4"
>>>>
>>>> --------------------------------------------------------
>>>> However, when I compile with 'intent(inout)', OR 'intent(inplace)', OR
>>>> 'intent(inout,c)' with appropriate F-/C-CONTIGUOUS arrays, then I get
>>>> the following error:
>>>> "pymm.error: failed in converting 4th argument `a' of pymm.mmfunc.mm to
>>>> C/Fortran array"
>>>>
>>>> --------------------------------------------------------
>>>> So, I want to know whether:
>>>>
>>>> 1) This could be due to a bug in the F2PY version I am using
>>>> [2.45.241_1926] that was later fixed?  Or a problem with my installation
>>>> of F2PY itself.
>>>>
>>>> 2) I am using a wrong flag (or not using a specific flag) during F2PY
>>>> compilation (please see CSH & LOG files in TGZ below)?
>>>>
>>>> 3) I am making some subtle mistake while calling the F2PY-module (please
>>>> see 'CALL_MM.PY' & 'MM.F90' file in TGZ)?
>>>>
>>>> Again, all the relevant text files (7 in all) are available at:
>>>> ftp://ftp.gps.caltech.edu/pub/rkanda/MemoryDeallocTest.tgz
>>>>
>>>> I don't know where else to look to try to resolve this problem.  So, it
>>>> will be much appreciated if someone can take a look at the F90, PY, PYF,
>>>> CSH, & LOG files, or actually compile/run the module to see whether  I'm
>>>> experiencing a "unique" problem here.
>>>>
>>>> Thank you very much for your time!
>>>> Ravi.
>>>> --------------------------------------------
>>>>
>>>> labrach wrote:
>>>>> It seems ok for me with the followings :
>>>>>
>>>>> (but I don't know how to check if the copy appeared or not, since the
>>>>> -DF2PY_REPORT_ATEXIT returns an error dealing with undefine reference
>>>>> to 'on_exit')
>>>>>
>>>>> I run that on a Windows with mingw 5.1.4
>>>>>
>>>> <SNIP>
>>>>
>>>> _______________________________________________
>>>> f2py-users mailing list
>>>> f2py-users at cens.ioc.ee
>>>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>>>>
>>> _______________________________________________
>>> f2py-users mailing list
>>> f2py-users at cens.ioc.ee
>>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>> --
>> ---------------------------------------------------------------------
>> Ravi Kanda
>> Seismological Laboratory, MC 252-21
>> Division of Geological and Planetary Sciences
>> California Institute of Technology
>>
>> 1200 E. California Blvd., Pasadena, CA 91125
>> Phone: 626-395-6971, Fax: 626-564-0715
>> Web Page: http://www.gps.caltech.edu/~rkanda
>>
>> ----------------------------------------------------------------------
>>
>> For a human being, the unexamined life is not worth living - SOCRATES
>>
>> ----------------------------------------------------------------------
>>
>> _______________________________________________
>> f2py-users mailing list
>> f2py-users at cens.ioc.ee
>> http://cens.ioc.ee/mailman/listinfo/f2py-users
>>
> 
> _______________________________________________
> f2py-users mailing list
> f2py-users at cens.ioc.ee
> http://cens.ioc.ee/mailman/listinfo/f2py-users

-- 
---------------------------------------------------------------------
Ravi Kanda
Seismological Laboratory, MC 252-21
Division of Geological and Planetary Sciences
California Institute of Technology

1200 E. California Blvd., Pasadena, CA 91125
Phone: 626-395-6971, Fax: 626-564-0715
Web Page: http://www.gps.caltech.edu/~rkanda

----------------------------------------------------------------------

For a human being, the unexamined life is not worth living - SOCRATES

----------------------------------------------------------------------



More information about the f2py-users mailing list