[reportlab-users] too many open files with inline images

[Ralf Schmitt]

Robin Becker wrote:

> Volker Haas wrote:

>> I get an error (too many open files) when using lots of inline images 

>> inside paragraphs.

>>

>> Sample code:

>>

>> from reportlab.platypus.paragraph import Paragraph

>> from reportlab.platypus.doctemplate import SimpleDocTemplate

>> from reportlab.lib.styles import ParagraphStyle

>>

>> import os

>> doc = SimpleDocTemplate('test.pdf')

>> elements = []

>>

>> for count in range(1024):

>>    print len(os.listdir("/proc/%d/fd" % os.getpid()))

>>    p = Paragraph('text <img src="math2.png" width="2cm" height="0.5cm" 

>> />' , ParagraphStyle('Normal'))

>>    elements.append(p)

>> doc.build(elements)

>>

>> OUTPUT:

>>

>> Traceback (most recent call last):

>>  File "test_fd.py", line 13, in <module>

>> OSError: [Errno 24] Too many open files: '/proc/32336/fd'

>>

> 

> I'm pretty sure this bug is not going to go away unless we rewrite the 

> ImageReader fp handling stuff. The intent is that ImageReader should be 

> able to handle a python file handle, a PIL image or a simple filename. 

> To avoid excessive work we open the image at the front rather than at 

> the back end. I suspect the only fix here is to delay opening(or reopen) 

> in the case when when a  path is given. In the cases when either a PIL 

> image or an open file is passed I don't think there's any obvious way to 

> cut down on the number of open files other than converting them into 

> memory based objects.

> 


Have a look at
http://code.pediapress.com/wiki/attachment/ticket/214/lazy-reader.diff

This implements lazy reading in case PIL is used and a file name is 
passed as argument.

While I'm here: The following makes reportlab's setup.py work with 
virtualenv:

~/reportlab/ hg diff -r 2c0ab57eba39 
 
ralf at red ok
diff --git a/reportlab/setup.py b/reportlab/setup.py
--- a/reportlab/setup.py
+++ b/reportlab/setup.py
@@ -5,20 +5,14 @@

  import os, sys, distutils, glob
  from distutils.core import setup, Extension
-
-# from Zope - App.Common.package_home
-def package_home(globals_dict):
-    __name__=globals_dict['__name__']
-    m=sys.modules[__name__]
-    r=os.path.split(m.__path__[0])[0]
-    return r
+from distutils.sysconfig import get_python_lib

  pjoin = os.path.join
  abspath = os.path.abspath
  isfile = os.path.isfile
  isdir = os.path.isfile
  dirname = os.path.dirname
-package_path = pjoin(package_home(distutils.__dict__), 'site-packages', 
'reportlab')
+package_path = pjoin(get_python_lib(plat_specific=1), 'reportlab')

  def get_version():
      #determine Version


and one more: lot's of assert in reportlab's code have no effect:

/home/ralf/py26/lib/python2.6/site-packages/mwlib.ext-0.8.2-py2.6-linux-x86_64.egg/mwlib/reportlab/pdfgen/pdfimages.py:108: 
SyntaxWarning: assertion is always true, perhaps remove parentheses?
   assert(len(raw) == imgwidth * imgheight, "Wrong amount of data for 
image")
/home/ralf/py26/lib/python2.6/site-packages/mwlib.ext-0.8.2-py2.6-linux-x86_64.egg/mwlib/reportlab/graphics/renderPS.py:586: 
SyntaxWarning: assertion is always true, perhaps remove parentheses?
   assert(len(rawimage) == imgwidth*imgheight, 'Wrong amount of data for 
image')
/home/ralf/py26/lib/python2.6/site-packages/mwlib.ext-0.8.2-py2.6-linux-x86_64.egg/mwlib/reportlab/graphics/renderPS.py:658: 
SyntaxWarning: assertion is always true, perhaps remove parentheses?
   assert(len(rawimage) == imwidth*imheight, 'Wrong amount of data for 
image')
/home/ralf/py26/lib/python2.6/site-packages/mwlib.ext-0.8.2-py2.6-linux-x86_64.egg/mwlib/reportlab/pdfbase/pdfdoc.py:2056: 
SyntaxWarning: assertion is always true, perhaps remove parentheses?
   assert(len(raw) == self.width*self.height, "Wrong amount of data for 
image")
/home/ralf/py26/lib/python2.6/site-packages/mwlib.ext-0.8.2-py2.6-linux-x86_64.egg/mwlib/reportlab/pdfbase/pdfutils.py:44: 
SyntaxWarning: assertion is always true, perhaps remove parentheses?
   assert(len(raw) == imgwidth * imgheight, "Wrong amount of data for 
image")




Regards,
- Ralf