PDF Export issue with file permissions

Help / Discuss
1

Update The FOP config didn’t provide a solution after placing it in any directory, but adding the user.home property to catalina.sh did. It’s important to add it to the JAVA_OPTS in the beginning of the file. I created a home folder (/var/tomcat/home) with the same permissions for _tomcat as the rest of the directories. This worked. Thank you very much again Enygma.

I believe a bug was created, but there has been zero movement on it for over a year.

Easily reproducible for me. Just take any page and attempt an Export As PDF. Error is at the bottom. While researching it, somebody suggested that the user running XWIKI should have a home directory accessible to it. As near as I can tell, that is the user responsible for the tomcat service. I’ve gone as far as giving 777 permissions to the home folder without fixing the issue.

This is a very critical error in XWIKI. We’ve created hundreds of nested pages for our documentation and have no ability to print them. The standard print-preview lacks the ability to include child pages, while the export can create a TOC and include attachments, comments, etc.

Without this ability we will have to abandon XWIKI and attempt migration to a different wiki platform, which will really hurt because we extensively evaluated the wikis out there and specifically chose XWIKI for many reasons.

Any help is sincerely and greatly appreciated.

Detailed information:
    Error number 11015 in 11: Exception while exporting
com.xpn.xwiki.XWikiException: Error number 11015 in 11: Exception while exporting
	at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:90)
	at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:446)
	at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:209)
	at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
	at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
	at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
	at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:112)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.xwiki.wysiwyg.filter.ConversionFilter.doFilter(ConversionFilter.java:127)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.xwiki.container.servlet.filters.internal.SetHTTPHeaderFilter.doFilter(SetHTTPHeaderFilter.java:63)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:208)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:111)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.xwiki.resource.servlet.RoutingFilter.doFilter(RoutingFilter.java:134)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1457)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.xpn.xwiki.XWikiException: Error number 12002 in 12: Exception while exporting pdf
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.createException(PdfExportImpl.java:493)
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.renderXSLFO(PdfExportImpl.java:253)
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.exportXHTML(PdfExportImpl.java:211)
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.exportHtml(PdfExportImpl.java:164)
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.export(PdfExportImpl.java:150)
	at com.xpn.xwiki.web.ExportAction.export(ExportAction.java:255)
	at com.xpn.xwiki.web.ExportAction.render(ExportAction.java:87)
	... 48 more
Caused by: org.apache.fop.apps.FOPException: .fop (Permission denied)
java.io.FileNotFoundException: .fop (Permission denied)
	at org.apache.fop.util.LogUtil.handleException(LogUtil.java:57)
	at org.apache.fop.fonts.FontCache.saveTo(FontCache.java:222)
	at org.apache.fop.fonts.FontCacheManagerFactory$FontCacheManagerImpl.save(FontCacheManagerFactory.java:71)
	at org.apache.fop.fonts.FontManager.saveCache(FontManager.java:142)
	at org.apache.fop.fonts.DefaultFontConfigurator.configure(DefaultFontConfigurator.java:103)
	at org.apache.fop.render.PrintRendererConfigurator.getCustomFontCollection(PrintRendererConfigurator.java:147)
	at org.apache.fop.render.PrintRendererConfigurator.setupFontInfo(PrintRendererConfigurator.java:127)
	at org.apache.fop.render.intermediate.IFUtil.setupFonts(IFUtil.java:170)
	at org.apache.fop.render.intermediate.IFRenderer.setupFontInfo(IFRenderer.java:187)
	at org.apache.fop.area.RenderPagesModel.<init>(RenderPagesModel.java:75)
	at org.apache.fop.area.AreaTreeHandler.setupModel(AreaTreeHandler.java:135)
	at org.apache.fop.area.AreaTreeHandler.<init>(AreaTreeHandler.java:105)
	at org.apache.fop.render.RendererFactory.createFOEventHandler(RendererFactory.java:363)
	at org.apache.fop.fo.FOTreeBuilder.<init>(FOTreeBuilder.java:107)
	at org.apache.fop.apps.Fop.createDefaultHandler(Fop.java:104)
	at org.apache.fop.apps.Fop.<init>(Fop.java:78)
	at org.apache.fop.apps.FOUserAgent.newFop(FOUserAgent.java:182)
	at org.apache.fop.apps.FopFactory.newFop(FopFactory.java:240)
	at com.xpn.xwiki.internal.pdf.FOPXSLFORenderer.render(FOPXSLFORenderer.java:133)
	at com.xpn.xwiki.pdf.impl.PdfExportImpl.renderXSLFO(PdfExportImpl.java:249)
	... 53 more
Caused by: java.io.FileNotFoundException: .fop (Permission denied)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
	at org.apache.fop.fonts.FontCache.saveTo(FontCache.java:213)
	... 71 more
2

Hi there. That’s sad but I believe that so far you’ve used XWiki for free and didn’t pay much for its features. Maybe it’s time to support the project and help it out progress more! You could do this by contracting with a company sponsoring the development of XWiki and ask them to fix the issue for you. This would benefit you but also the XWiki project. Another option is to submit a fix yourself but you probably don’t have the knowledge or you’d have done it already I guess :slight_smile:

See https://www.xwiki.org/xwiki/bin/view/Main/Support#HProfessionalSupport

5

I believe a bug was created, but there has been zero movement on it for over a year.

Where? Can you give a link? PDF export works fine for me and there are even automated tests to prove that PDF export works with every release of XWiki.

If you’re referring to Pdf export issue - #10 by mattyp and Loading... then there’s already a solution:

make sure that the user that is running xwiki has a home directory, because the pdf export creates a cache in ~/.fop

If that didn’t work for you then you should check the code of Apache FOP because the code that triggers your problem is not in XWiki, as shown by the stack trace you pasted

at org.apache.fop.fonts.FontCache.saveTo(FontCache.java:213)
6

Thank you very much for your reply. I appreciate constructive comments, and help, not caustic cheeky comments asking for money.

The bug (14927) does not have a solution. It’s listed as Major, Open, and Unresolved as of this writing. If you read my original post, I already attempted to implement the solution recommended in the bug comments.

Thus far the solution hasn’t worked, and if it really does work, then somebody should update the bug status to “fixed” or “closed”, or whatever is appropriate.

As for the stack traces, I don’t understand JAVA error messages, nor am I a JAVA programmer. I have a limited understanding of what that means.

I already grep’d all of the source code under /var/tomcat/webapps/xwiki and was unable to find any lines of code anywhere. Ostensibly it is line 213 in some module, but I cannot find it.

Is Apache FOP installed separately on my machine or something, and not part of the WAR distribution for XWIKI?

That’s why I’m asking for a little direction here. I cannot find that cache setting anywhere, in any file under /var/tomcat.

I’ve also created the home folder for the user running XWIKI, which I assume is the one responsible for Tomcat correct? As I said above, I already set 777 permissions on that home folder trying to see if that would work at all.

Any help to configure Apache FOP is very appreciated, especially where to configure it in XWIKI’s code base.

7

I cannot find any lines of code or configurations with the setting. I used the following search command in /var/tomcat

find ./ -type f -exec grep -E "cache" {} \; | grep -E "fop|fonts"

Are you guys embedding Apache FOP into the XWIKI distribution, or do you expect it installed and configured on its own as a dependency?

8

After investigating Apache FOP, I have a few questions:

  1. Are the configurations for Apache FOP performed programmatically in XWIKI, or are they contained in an .xml file?
  2. If yes, where inside the code base for XWIKI are these settings? Specifically use-cache and cache-file
  3. If no, where inside a WAR distribution is the xml file contained?

If it isn’t appropriate to configure Apache FOP within XWIKI, but inside the overall Tomcat installation, where might I configure Apache FOP for all web applications being hosted?

9

Hi,

I’m really not up to date with how PDF export/FOP works, but if you really want to go the way of trying to change the default FOP cache location, maybe this can help:

As far as I can see, we do have a “default” configuration file (provided inside xwiki-platform-oldcore-*.jar) which then also gets extended with some font configuration auto-detection performed by XWiki.

This is the current content of the file xwiki-platform/xwiki-platform-core/xwiki-platform-oldcore/src/main/resources/fop-config.xml at master · xwiki/xwiki-platform · GitHub and you can make your changes by dropping your modified version of this config file in tomcat/webapps/xwiki/WEB-INF/classes/fop-config.xml and probably restarting xwiki (tomcat, to be more precise).

For future reference, if I read well, the FOP configuration loading seems to be handled around these lines xwiki-platform/xwiki-platform-core/xwiki-platform-oldcore/src/main/java/com/xpn/xwiki/internal/pdf/FOPXSLFORenderer.java at master · xwiki/xwiki-platform · GitHub

Additional configuration options on the Apache FOP website: Apache(tm) FOP: Configuration

Additional details on how to customize PDF exports (and maybe understand better how XWiki uses FOP, if needed): https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Configuration/#HCustomizingthePDFexportLook26Feel

Regarding your actual issue, it sounds more like a configuration problem that not that many users face and that is not really specific to XWiki (as previously stated). However, please report back if you succeed in setting a custom cache location, as it may be an option for XWiki to try to force a cache location (if not already specified in the config) to a folder that is writable (either temp folder or permanent directory), similarly to how the FOP configuration is extended with font locations, to try as much as possible to avoid this problem from surfacing.

10

Thank you very much for the informative reply Enygma.

I don’t actually have xwiki-platform-oldcore-*.jar. These are not present. However, xwiki-platform-legacy-oldcore-9.11.2 exists and contains the fop-config.xml.

As instructed, I dropped the modified version of fop-config.xml in /var/tomcat/webapps/xwiki/WEB-INF/classes. The added lines of configuration look like this:

<fop version="1.0">
  <use-cache>true</use-cache>
  <cache-file>file:/tmp/fop-fonts.cache</cache-file>

After restarting tomcat, the problem was not fixed. I’m not sure about the syntax for the cache-file though. The example from StackOverflow has file:/Directory and I don’t know if that is a replacement variable, or literal. Confirmation of correct syntax would be nice.

In reading the code that you linked to I was unable to determine what directory it was actually using.

private static final String FONTS_PATH = "/WEB-INF/fonts/";

I don’t know if that is the path that is being denied, but I can confirm that the directory in question has full permissions for the _tomcat user. I even gave it 777 for good measure to make sure it wasn’t a permission issue. On that note I’m sorely tempted to just apply 777 on the whole damned box and see what happens :slight_smile:

On another note, and this is just an opinion here, but JAVA stack traces are wholly useless. I’m used to Perl, where it will tell you not only the line of code causing the issue, but the exact file causing the issue. Furthermore, the stack traces are insufficient in actually telling me what directory XWIKI is trying to use. Which diagnostically, makes it pretty gosh darned hard to verify it is actually using the directory I specified.

Is there any way to vastly increase the logging detail, and would that show which directory it is trying to use? Save that, do you think it would be appropriate to modify the code in renderer to add diagnostic lines to output the directory being used? In other words, I’m willing to attempt to write that code and contribute it back to the project.

I saw your comment on the bug, and I wholly agree. This is extremely ridiculous and it seems like no solution is possible on my server. I. Just. Don’t. Get. It. Still, your idea is appealing and I’m interested in diving into the JAVA code to fix this.

P.S - I put the fop-config.xml in /classes as well the /var/tomcat/webapps/xwiki directory too. Just for #&#$% and giggles, I also updated the fop-config.xml in the jar file with my cache settings above, but no joy on that.

Thank you very much again for you help.

11

Java stack traces give you the Class file, line and number where an exception is throw, together with the message of that exception. If the exception message is not explicit enough as to specify the files/paths it failed to access, then it’s fault of the particular dev, not the language itself :slight_smile:

Back to the problem… Looking at the Apache FOP code around this configuration (xmlgraphics-fop/fop-core/src/main/java/org/apache/fop/fonts/FontManagerConfigurator.java at bd7d5048785c691e6e3e152af10805f3127b760d · apache/xmlgraphics-fop · GitHub), I see 2 options:

  1. Disable font cache completely (<use-cache>false</use-cache>) to avoid any attempts of writing to a cache folder
  2. Leave caching enabled, but make sure to use a correct URI for the folder resource on your machine. File URIs are in the form file://, followed by the path on your machine (which also starts with one / on linux). That would mean <cache-file>file:///tmp/fop-fonts.cache</cache-file>

Looking more at the FOP code that works with font cache xmlgraphics-fop/fop-core/src/main/java/org/apache/fop/fonts/FontCache.java at 48dc3b2dafd481af00f5cc9ee706115216c4e85a · apache/xmlgraphics-fop · GitHub (yes, I got here from the java stack trace! :slight_smile: ) and the actual exception FileNotFoundException: .fop (Permission denied), it means that getDefaultCacheFile(true) resolves the generic .fop directory (line xmlgraphics-fop/fop-core/src/main/java/org/apache/fop/fonts/FontCache.java at 48dc3b2dafd481af00f5cc9ee706115216c4e85a · apache/xmlgraphics-fop · GitHub) which is an absolute fallback because the user home directory was not usable (i.e. userHome was null and this if failed to enter xmlgraphics-fop/fop-core/src/main/java/org/apache/fop/fonts/FontCache.java at 48dc3b2dafd481af00f5cc9ee706115216c4e85a · apache/xmlgraphics-fop · GitHub because getUserHome() returned null). I am deducing this, because, otherwise, your exception should have looked more like FileNotFoundException: /home/tomcat/.fop (Permission denied) or FileNotFoundException: /tmp/.fop (Permission denied). It should actually include the font cache file itself as well, e.g. /home/tomcat/.fop/fop-fonts.cache… you get the picture.

So the root of the problem seems to be an issue with the running java environment in figuring out the home folder of the user executing it. In lack of a user home to prefix the path, simply .fop would resolve to a subfolder of the current working folder (i.e. the value of the system property “user.dir” of the running java environment; i.e. the place where the command to start the tomcat instance was executed from). You could try to debug the “current folder of the running instance” by executing this quick script in one of your XWiki test pages:

{{groovy}}
System.getProperty("user.dir")
{{/groovy}}

(hint: you can do the same for the user.home property as well to actually debug the running instance’s setup. To get all properties, use System.getProperties() instead and find the interesting property values there.)

You might get some good debugging info (but from outside the running instance) by executing sudo -u tomcat java -XshowSettings:properties -version. It should print out useful information (would be useful to share it here as well), specifically at the end, the value of user.home. Make sure to specify the actual user that is running your tomcat instance (you seem to have mentioned your user is called _tomcat and not tomcat).

If you that does not help, your backup is to force the value of the user.home in your java environment by tinkering with the tomcat initialization scripts (well documented online, either setenv.sh or directly catalina.sh, don’t remember exactly) and add the system property -Duser.home=/home/_tomcat (or whatever you prefer), just note that there might be other things (not only FOP) relying on that system property and you might affect them as well.

This should be enough to help you figure out what is wrong in your setup… either with the user that is running the java env, either with the java env itself or with rights in your system. Finally, you have 2 ways of overriding the settings (in the fop-config file or with the java user.home system property).

Not more I can do from here at this point. Hope you come back with a positive outcome.

12

I got it fixed. Sort of. Can’t remember what I did specifically that fixed it, but I will repeat it tomorrow and isolate it. I’ll write something up here.

That being said, it was all rather pointless. Export As PDF is incapable of printing children and ignores the hierarchy selections.

http://jira.xwiki.org/browse/XWIKI-12544

From a comment there by the superhelpful vincent massol, I can see that it was never intended to. Or at least it doesn’t seem to be proposed as a feature.

Back to square one for printing everything.

13

caustic cheeky comments asking for money
…
From a comment there by the superhelpful vincent massol

A couple of old sayings come to mind: “Don’t bite the hand that feeds you” and “It is easier to attract flies using honey than vinegar.” FWIW I would recommend “softening” your approach regardless of how frustrated you are. Keep in mind that the XWiki team provides this software as open source.

14

bstewart, I can appreciate your comment, but vincent was out of line. Pushing paid support in a forum is inappropriate , especially as the first step. His comments were not constructive in any way possible. Considering his comments on the feature itself (what I linked to), he was the most cognizant of exactly what is going on here, and has been 0% helpful. He could have said right away that printing all of the children in a PDF wasn’t possible with the export tool, at least through the GUI, as he was the one proposing not to do it.

Considering all of that, vincent deserved every comment I made, and more. If he is absorbed with money, then he needs to go some place to make money. I respond to vinegar with vinegar, forgive me.

Others have been very, very, helpful which is appreciated. As you can see, I’m dedicated to figuring out this problem and contributing whatever information and code I have back to XWiki. So if there are any hands doing the feeding right now, they belong to myself and Enygma.

I’m not at the office yet, but I will be isolating what actions fixed my specific issue with FOP, and then I will begin working on whatever is required to perform multi-page PDF printing that handles children. That is a core critical component, and it is missing. Right now printing in Xwiki is rather incomplete.

So when I get done:

  1. Problem fixed, and documented
  2. A how-to on installing and maintaining Xwiki on OpenBSD environments. (This specifically seems to be an issue since the default tomcat installation from packages doesn’t fully or correctly provision the user.home property)
  3. Multi-page PDF export with support for printing children.

P.S - In my project right now, paid support for open source projects is simply too premature. However, when it gets off the ground and starts making money, we fully intend to set aside a portion of profits to obtain support contracts for our core open source components. XWiki would part of about a dozen that we would want to support. One of the great things about Open Source is that you can get started without huge barriers to entry, but that means people help each other and contribute. I’ve already had enough of some other fake open source projects that demand money (thousands) and refuse to answer any questions. Vincent is a perfect reminder of such people with such attitudes.

15

vincent was out of line.

We will have to agree to disagree about that. I don’t think his comments were inappropriate.

16

Another option is to submit a fix yourself but you probably don’t have the knowledge or you’d have done it already I guess :slight_smile:

That’s appropriate and constructive, and inline with the forums own guidelines? We will have to agree to disagree on this one.

17

I’ve already had enough of some other fake open source projects that demand money (thousands) and refuse to answer any questions. Vincent is a perfect reminder of such people with such attitudes.

Vincent has helped a number of people on this forum (including me!) with XWiki issues, and he has done this for free and without demanding anything. He is one of the important people on the XWiki project that has made it successful. As has already been noted - nobody is forcing you to use XWiki. It’s obvious that this specific issue has caused you some frustration, but if you want free help, my recommendation (as I noted before) is not to rant and rage. You are free to disregard this advice of course, but you may find that others might be less willing to be accommodating in the future.