Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Welcome to the CollectiveAccess support forum! Here the developers and community answer questions related to use of the software. Please include the following information in every new issue posted here:

  1. Version of the software that is used, along with browser and version

  2. If the issue pertains to Providence, Pawtucket or both

  3. What steps you’ve taken to try to resolve the issue

  4. Screenshots demonstrating the issue

  5. The relevant sections of your installation profile or configuration including the codes and settings defined for your local elements.


If your question pertains to data import or export, please also include:

  1. Data sample

  2. Your mapping


Answers may be delayed for posts that do not include sufficient information.

Cannot upload bigger images

For some reason, I can only upload small photos and scans to Providence. Bigger photos do seem to be uploaded and some processing seems to take place, but then the files are never added as media representations. I only tried images in jpeg format and for that format I only seem to be able to upload files that are smaller than about .5Mb. I am using Providence 1.7.6, but my problem occurred with 1.7.5 as well. My provider is DreamHost and the system settings for PHP (7.0.30 with FastCGI) seem fine (max_upload_filesize=64M and post_whatever = 64M as well). All image processing software comes installed by default as well. Could this be a metadata setting for MySQL perhaps as I read somewhere? Under http, I do get an error message in my Apache error log:

(28)No space left on device: mod_fcgid: can't write tmp file for stdin request, referer: http://****.org/CollectiveAccess/index.php/editor/objects/ObjectEditor/Edit/Screen40/object_id/82

Not sure what this means as there is plenty of space left. I do not get this error message when using https, but the behavior is the same. Any suggestions are welcome.

Thanks,
Eisso

Comments

  • Probably the partition where temporary files are written is nearly full and/or quite small.

  • Thanks Seth. You could very well be right. Dreamhost uses a 128M RAM-based tmpfs for each VPS and when I checked it seemed pretty full (98%). Just to be on the safe side I rebooted and after that I was able to upload a slightly bigger file (about .7M), without leaving any files in /tmp. Next, I tried to upload a 3.3M file (uncompressed: 44M) and that failed. When I did "ls -la" on /tmp, it was clear that this failed process did not clean up after itself very well either:

    -rw------- 1 *** pg2573328 97881992 Sep 22 06:25 magick-AP2mUtLf
    -rw------- 1 *** pg2573328 97881992 Sep 22 06:25 magick-eKq2SNJm
    -rw-r--r-- 1 *** pg2573328 3294222 Sep 22 06:25 phpYORaPO.jpg

    The third line refers to the original file. In addition, ImageMagick/Imagick left two files each a little over twice the size of the uncompressed original. This seems a little bit strange. Why two files? I also thought imagemagick only needed about three times the uncompressed size of the uploaded file? (Or was that server memory?) The two files combined take up more than 4 times the uncompressed size of the original, which incidentally is a lot more than the 128M that /tmp has, but df still says /tmp is 98% full. Some swapping going on? For the same time stamp, the Apache error log has the following two lines:

    [Sat Sep 22 06:25:24 2018] [warn] [client 24.198.100.125] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: http://www.theoldtownmuseum.org/CollectiveAccess/index.php/editor/objects/ObjectEditor/Save/Screen40/object_id/82
    [Sat Sep 22 06:25:24 2018] [error] [client 24.198.100.125] Premature end of script headers: index.php,
    referer:http://www.theoldtownmuseum.org/CollectiveAccess/index.php/editor/objects/ObjectEditor/Save/Screen40/object_id/82

    What is going on here exactly? As I said at the beginning, you are probably right that 128M of /tmp is too small for a 600x600dpi scan of a standard size photo (which is what the 3.3M file was).

    Thanks again, Eisso

  • ImageMagick has to uncompress the JPEG to process it. The uncompressed version is going to be much larger than the initial JPEG. Depending upon the exact processing, it's going to make 2 or more copies of the uncompressed version. So yes, you need more disk space.

    One more thing to try is:

    1. Create a temporary directory somewhere on the server where there's more disk space.
    2. Make sure the web server can read and write to that directory. If you're not sure who the web server "user" is just set it to 777 (world writable) for now. If it works that way then you can muck with permissions later.
    3. In your setup.php file add the following line: putenv("MAGICK_TMP=/path/to/you/new/tmp/directory");

    This should force ImageMagick to use your new temporary directory rather than /tmp

  • Sounds great, but I cannot get it to work. Per your suggestions, I created a tmp directory /home//tmp, where /home/ is my home directory, and set its permissions to 777. Next, I put the line

    putenv("MAGICK_TMP=/home/***/tmp");

    at the beginning of setup.php. I also put the line

    upload_tmp_file=/home/***/tmp

    in my phprc and then rebooted. All to no avail. According to phpinfo(), PHP notices the new upload_tmp_file path, but nonetheless what actually happens when I try to add a representation is that the image file is still uploaded to /tmp and imagick still dumps its two big files there. In addition I now get the onscreen error "no input file was specified". The Apache error messages are still the same. Something is not right...

    Any help would be appreciated...
    Thanks, Eisso

  • It's not PHP that takes this setting. It's ImageMagick. I have a feeling it's not "taking" because you're running under FCGI (I'm guessing that because of your mention of phprc), but that's just a guess. You might want to ask your host how to set environment variables (eg. MAGICK_TMP) for all processes. It's possible but typically requires root on the machine.

  • Thanks, I have taken this up with Dreamhost. Let's see what they come up with. I will let you know. Eisso

  • Having Dreamhost move /tmp to my local drive solved the problem. I can now upload 3M files without any issues. A minor issue, though, is that Imagemagick still does not clean up after itself and leaves two rather big files in /tmp. Not a real problem though as adding deletion of any /tmp/mag* files that are more than two days old to my daily cron backup script should fix this. It just seems a little sloppy. Also, I have not checked again, but I seem to remember that this does not happen for smaller files. Maybe I am just dreaming.
    Thanks again,
    Eisso

Sign In or Register to comment.