Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the CollectiveAccess support forum! Here the developers and community answer questions related to use of the software. Please include the following information in every new issue posted here:

  1. Version of the software that is used, along with browser and version

  2. If the issue pertains to Providence, Pawtucket or both

  3. What steps you’ve taken to try to resolve the issue

  4. Screenshots demonstrating the issue

  5. The relevant sections of your installation profile or configuration including the codes and settings defined for your local elements.


If your question pertains to data import or export, please also include:

  1. Data sample

  2. Your mapping


Answers may be delayed for posts that do not include sufficient information.

Large Video Upload Memory Issues

Hello,

I am hosting a CA 1.6.1 installation on an Ubuntu 14.04 server.  My use case for CA is to catalog a large collection of home videos that have been digitized to mp4 format.  These files are stored on an smb share.  Some of these videos are relatively large (350-ish mb) and in those cases uploading them will cause php memory errors.  I have updated the php values for upload_max_filesize and post_max_size to 500M and memory_limit to 512M but it isn't enough.  Additionally, we will have videos around 1gb in size down the road so I'm hesitant to keep raising php's memory limit as a solution since our resources are limited. 

I am wondering if there is a way to avoid the upload process entirely by manually assigning a ca_object to its associated video, maybe by providing a file path directly to the file.  We would still want to be able to download and stream the video through providence.  Please let me know if you have any other ideas or suggestions and thanks in advance for your help.

Comments

  • Hi,

    It shouldn't need that much memory to process a video. Images tend to have high peak memory requirements because operations require them to be decompressed into memory. Video is processed in a stream and has a much smaller footprint. (I'm watching a 400meg FLV file process right now and it's not using much of anything in the PHP process, although ffmpeg has grabbed a good chunk of memory for itself).

    When you get memory errors where does the process die exactly?


    seth 
  • Hmm...it looks like our PDFWand plugin is getting triggered when trying to determine the upload's file format.  This is then causing Zend_Pdf_Parser to try to open the video as a pdf which would explain why it's being loaded into memory.  Not sure why the PDF plugin is being triggered at all but at least I know the source of the issue.
  • Found the solution.  For anyone else who might be having the same problem it looks like it was resolved here: https://github.com/collectiveaccess/providence/commit/e81b812f53f5c6321a3404dd5480b08fbe773f3b

    Thanks for your help seth.
  • Yeah, should've thought of that one :-)
    Zend_PDF can be very handy for certain types of PDFs, but it's not that smart about bailing on files that are clearly not PDFs.
Sign In or Register to comment.