PmWiki | Cookbook / PublishPDF

Summary: Typesets wiki pages and page collections into PDF (finalist: New Zealand open source awards 2008)

Version: 2.2.08 (released 2009-01-01) -- see release notes, describing enhancements and minor improvements to the PDF documents

Status: Stable, reliable and substantially complete

Prerequisites: pmwiki 2.1.27 and above, tested and stable with pmwiki 2.2beta65

Prerequisites: (if you are running a local Wikibook PDF server) Server 1.0.0 or above (available from http://www.wikipublisher.org/) -- version 2.2.08 gives access to features added in server version 1.0.8

Maintainer: jr

Categories: Publishing, PDF, Bibliography, Equations, Custom PageStore

Question

How can we use a wiki as a collaborative desktop publishing system, to generate publishing-quality PDF files from any wiki page collection?

Why do we want to do this? Reading long documents on screen is still a poor substitute for reading the same content on paper. Word processor output is of higher quality than a printed web page, and the output of high end publishing software is better still. Using a wiki as a collaborative publishing environment ensures consistency of published output, allowing authors to focus on content rather than being distracted by presentation matters.

Answer

The Wikipublisher.BasicFeatures.pdf Δ, produced from a standard PmWiki page, summarises and illustrates what the library offers. It typesets text, lists, tables, images and equations ready for printing. It provides citation markup to support bibliographies. See http://www.wikipublisher.org/ and give it a try.

Next step...

Install the PublishPDF library

download and expand wikipublisher-2208.tar.gzΔ (current stable version)
- this version is robust and stable under PmWiki 2.1.27; testing under PmWiki 2.2beta65 is complete and all known problems have been fixed
- if you are running Wikipublisher and upgrading from PmWiki 2.1.27 to 2.2, please read Wikipublisher.UpgradingToPmWiki22
browse through the sample Wikipublisher.UserGuide.pdf.zipΔ (100kB zipped; 184kB pdf) (generated using the wikipublisher library)
- current versions are available at Wikipublisher.UserGuide; see also TipOfTheWeek
move the skin directories from skins/ into pub/skins/
move the pdf icons from images/ into pub/images/
move the css files from css/ to pub/css/
move the cookbook/wikipublisher/ directory into cookbook/
- moving wikipublisher/ will move various scripts plus a latex/ directory into cookbook/
add include_once("$FarmD/cookbook/wikipublisher/extensions.php"); to the start of local/config.php
- you can override the default wikipublisher skin by setting $Skin = 'yourskin'; after the include
- it's best to test wikipublisher with its default skin first
disable MarkupExtensions, PageTableOfContents, PublishWikiTrail, RenamePage, SearchExtensions, WikiForms and WikiCalendar (all included in the PublishPDF recipe)
- the versions included with wikipublisher are tested as a unit; use other versions at your own risk
disable any of the DictIndex versions you may have installed (the PublishPDF version can be disabled if preferred; set $MarkupEnabled['titledictindex'] = 0;)
register your site at http://www.wikipublisher.org/
try a search -- you should see a PDF checkbox and a Publish button
you may wish to disable one or more of the optional recipes -- pagetoc, rename, search, wikilog -- by setting $MarkupEnabled['recipe'] = 0; where 'recipe' is 'pagetoc', 'rename', 'search', or 'wikilog'

Now add (:typeset:) to a WikiTrail page. You should see a Typeset button and a PDF options link.

You can also run your own wikipublisher pdf server: InstallTheServer

To get help from (and in turn help) other users, join the DiscussionGroup.

Print any combination of wiki pages

The intent is to support a print run of one. Provide a one-click publishing button that can be added to any of the following:

a single wiki page
a trail page
selected stops on a trail
a search results page
a category page
a backlinks page (the links to the current page)
a links page (the links from the current page)
any other dynamic page list, such as WikiCalendar, WikiForms, and so on

For dynamic page lists, such as search results, the visitor can choose which pages to include in the output. Each page in the list has a checkbox to include it in the output.

Apply print-oriented styles

Pressing the publish button will typeset the selected pages in a form suitable for printing or distributing as a PDF. This includes:

a cover page
a summary page
a table of contents page
PDF bookmarks and metadata
internal cross-references (such as see page xx)
running headers and footers (including mirroring for duplex printing)
substituting high resolution versions of images, if these are available
auto-numbering of major headings
translating all wiki markups to their print equivalents
footnotes for references to external links
etc, etc, etc

Control print presentation

The information seeker has various options to change the look of the generated PDF. These include:

what the cover looks like (title, alignment, font, etc)
duplex printing
heading and body text styles
paper size
section numbering
watermark text

The print presentation options are defined in Site.Template pages (included in a wikilib.d/ with the distribution and automatically added to the list of page locations). The publishing options cascade:

run-time publishing settings take first priority
if no run-time settings, a Group.Template takes priority
if no Group.Template, a Site.Template is used

There are different templates for typesetting a single page, a trail page, a calendar page, and a search results page. An administrator can edit these like any other wiki page, for example to translate the form options into a different language. These pages use the WikiForms markup syntax.

History

See the release log

01 January 2009 -- 2.2.08 enhances custom floats and supports width and align in advanced table cells
02 November 2008 -- 2.2.07 enhances Fig, Tab and Div cross-references, if $ReferencePageNumbers = true;
10 October 2008 - 2.2.06 enhancements to tables and cross-references to anchors
23 June 2008 -- 2.2.05 minor enhancements to footnotes and monospaced text
20 June 2008 -- 2.2.04 cosmetic enhancements to improve book and report layouts
22 May 2008 -- 2.2.03 cosmetic enhancements to improve book and table layouts
19 April 2008 -- 2.2.02 minor enhancements to table handler and some bug fixes
10 March 2008 - 2.2.01 includes fixes to support PmWiki 2.2, adds a floating div option (treat a div like an image or table) and <hr> print style
30 January 2008 -- 2.1.11 is tested with PmWiki 2.2beta65 and improves presentation of images and tables
23 November 2007 -- 2.1.9 adds support for rotating images and tables
06 November 2007 -- 2.1.8 maintenance release enhances equations, list styles; adds highlight, quotation markup
10 October 2007 -- 2.1.7 maintenance release with minor enhancements, notably in handling of long Urls addresses
20 September 2007 -- 2.1.6 a maintenance release that fixes minor bugs
06 September 2007 -- 2.1.5 create and typeset letters, with addressee, subject, salutation, return address and so on
04 July 2007 -- 2.1.4 improves support for citations and bibliographies
09 May 2007 -- 2.1.3 adds support for citations and bibliographies
23 March 2007 -- 2.1.2 is mostly a maintenance release, preparating for citation support
07 December 2006 -- 2.1.1 tidies support for nested div markup and fixes minor bugs
06 October 2006 -- 2.1.0 adds wikibook template
20 July 2006 -- 2.0.9 various minor bug fixes and improvements
11 July 2006 -- 2.0.8 support for figure and table cross-references, comment blocks
27 May 2006 -- 2.0.7 improve treatment of headings
12 May 2006 -- 2.0.6 minor improvements and bug fixes
13 April 2006 -- 2.0.5 maintenance release that addresses minor bugs
03 April 2006 -- 2.0.4 fix bug arising from whitespace markup in PmWiki 2.1
01 April 2006 -- 2.0.3 minor feature enhancements and bug fixes
06 March 2006 -- 2.0.2 add 2 column support and improve image logo support
23 February 2006 -- 2.0.1 fix $EnableWikiStyles problem with pmwiki 2.1beta26
21 February 2006 -- wikipublisher server released
17 February 2006 -- 2.0.0 version released with PmWiki 2.1 compatibility
02 February 2006 -- 2.0 beta 9 released, long table improvements, final beta, server-side to follow
21 December 2005 -- 2.0 beta 8 released, more style improvements
16 December 2005 -- 2.0 beta 7 released, improves wikistyle handling
22 November 2005 -- 2.0 beta 6 released
27 October 2005 -- 2.0.beta5 released
07 October 2005 -- 2.0.beta4 released
23 September 2005 -- 2.0.beta3 released
12 September 2005 -- 2.0.beta2 released

Notes and Comments

This recipe is fully tested on PmWiki version: 2.1.27; there are no known remaining incompatibilities with the new features and changes introduced in PmWiki 2.2, as tested with version 2.2.0 beta 65. To use the Publish PDF library with the wikipublisher.org typesetting server, your wiki must be accessible over the web. To use it with a private wiki, you must set up your own local typesetting server. Recipe version 2.1.0 or above requires server version 0.9.4a or above. Recipe version 2.1.3 or above requires server version 0.9.5a or above.
Conflicts with cookbook recipes extendmarkup.php, extdictindex.php (includes its own version of these, tested with the pdf server). See also Wikipublisher/UpgradingToPmWiki22, for a discussion of pagelist and transition issues.

The recipe comes in 2 parts:

a library of PmWiki scripts which instruct PmWiki to generate output using a print-oriented xml dtd; this includes suitable skins as well as replacements for the stdmarkup, wikistyles and other scripts -- these are installed as a PmWiki recipe
a set of server scripts which transform the print-oriented xml output to a typeset pdf and download this to the reader's desktop -- these can be installed on any server anywhere (available from wikipublisher)

We have set up the pdf server as a web service and the PmWiki recipe points to it. An administrator only has to install the PmWiki recipe. An administrator may choose to run the pdf server software as well, for example to support publishing behind a firewall, or if the webserver uses a port other than 80 (our hosting service restricts outbound connections to port 80).

The main site is at http://www.wikipublisher.org/wiki. This is a working and fully-functional demonstrator.

So can i try it out? Where is the attachment? DirkBlaas

The first beta release is available now. The wikipublisher server is up and running; the code for the PmWiki recipe and server-side typesetting is complete and tested. The above Demo is stable and reliable. jr

I've tried it but it horribly nesses up me custom skin. even on standard pages. What should I do ?

Provide a url where we can have a look and expand on 'horribly messes up'. What is going wrong and what do you want to see happen? Are we talking about the logo? Wikipublisher uses a text logo (not an image) by default. Are we talking about what the pdf icon on every page does to the page? If so, this is what the library does by default. We insert $PageTypesetFmt into $HandleBrowseFmt ahead of $PageRedirectFmt. There is a p.pdf style that does 'float: right; margin-left: 8px;'. This puts the pdf icon into the top right hand corner of the page text. More information, please.

To be more precise, (and I know this is still a beta), the recipe seems to replace my custom skin with its own, which is not acceptable. Is there something that I am doing wrong ?

The recipe sets a default skin, very similar to the pmwiki default skin, which fully supports all wikipublisher features. It is a good idea to test your wikipublisher install with this skin first, to make sure it is working correctly. If you then add a $Skin = 'yourskin'; statement to local/config.php after the statement to include wikipublisher/extensions.php, your skin should load.

The following comments refer to the beta series of releases. The 2.1.x and 2.2.x series avoid these issues. jr

This seems to be a very interesting recipe, but there are a number of problems with it:

Installation: the list above (and on the site) is incorrect and not complete:
- It contains a latex directory that does not seem to go anywhere
  - just move the wikipublisher/ directory into your cookbook/ directory; the wikipublisher/latex/ directory will move along with it -- it may not work if the latex/ directory is placed anywhere else
- It seems to expect the scripts to go into a scripts directory in the cookbook, not a wikipublisher one.
  - no, it expects the scripts to go into cookbook/wikipublisher/ -- just move the wikipublisher/ directory into cookbook/
- It is annoying to have to disable other extensions to run it. Why should they be included, especially if the version included is older ? It seems that the disabling option listed above for the DictiIndex actually works for most of them (like the calendar), if you look at the code. It would be nice to have the list.
  - the versions of recipes included are tested to work correctly as part of wikipublisher; sometimes, individual recipes move ahead, but the next wikipublisher release will catch up -- it's my intention that the preferred versions are those that make up wikipublisher, but at the moment we are in transition; it seemed better to include everything required to run wikipublisher, rather than asking people to download other recipes as well; check the documentation on how to disable the optional recipes -- see Wikipublisher.Recipes
It replaces my defined skin by its own. I can understand that for printing, it should do this, alright. But not for the consultation of the wiki.
- The recipe sets a default skin, very similar to the pmwiki default skin, which fully supports all wikipublisher features. It is a good idea to test your wikipublisher install with this skin first, to make sure it is working correctly. If you then add a $Skin = 'yourskin'; statement to local/config.php after the statement to include wikipublisher/extensions.php, your skin should load. At a guess, your $Skin = statement fell before the wikipublisher include statement (fixed in version 2.0.5).
Although my site is registered on yours (newgallo.dnsalias.net), it still does not seem to work. I might have the firewall problem (see below), but it is not clear from the error message.
- The server-side is available for download and local installation. When you make a pdf request from behind a firewall, pmwiki will try to visit a site (the wikipublisher server) that is not accessible. Presumably, the web server running pmwiki then issues some kind of error message. Is there a way that we can detect you are behind a firewall and issue a more suitable error?

In any case, I have disabled it for now, but I very much look forward to see the improvements.

Thank you for this excellent feedback. I hope the above responses are helpful and clarify matters. jr

I've installed the software as described but I'm behind a firewall and it doesn't work. Think I need the 'server-side' pdf convertion software - where can I get this from? Smc

If you are behind a firewall, then the typesetting server can't request the xml from your web server. There may be some things you can do to allow your pmwiki server to communicate with the typesetting server through the firewall. You'd need to discuss this with your technical staff. The server side software relies on a number of prerequisites (for example, I believe Debian Linux includes most of what you need):

Latex
ImageMagick
an xslt processor, such as Libxslt or Saxon (we found this was quite slow)
access to your own PmWiki approved url page

You may wish to test the pdf service on a public web site first, rather than going to a lot of effort setting up a server and then perhaps finding that it's not suitable for your needs. We haven't yet documented the server-side install process, so some patience and perseverence will be needed. Send an email to the pmwiki discussion list or directly to John Rankin (address in the software readme file) and we'll work out the best approach. Can you describe your operating environment in a bit more detail? jr

John, thanks for your response. Very interested in pursuing the server-side installation. Our company does not permit external public systems from accessing our internal network and documents. I've been tracking your project for a while as it seems to have the edge on other variants with it's intelligent mining of information (e.g. Wiki trail) and the high quality visualisation enhancing readability. This is why I feel its worth investing set-up time in this. I would be forced to setup an end-to-end solution from within the confines of our intranet to ensure no document 'left' our building. As you suggest, I'll contact you on the news group to see how we can proceed on this. Smc

The server software is sufficiently robust for people who want to give it a try, available from Wikipublisher.InstallTheServer. jr

Comment: i've just installed it (the server), works nice, but i have some problems :/

 Q: How to make it use only localhost communication?

What does this question mean? What is the set-up and behaviour is wanted?

 Q: How to make it include pmwikidraw ?

You need a publishing markup rule that re-interprets the draw files as regular Attach: ... .gif directives, I think. See Issues.00063.

Is it possible turn off linking to web sites in the generated PDF file?

Yes. This is controlled with PmWiki's standard link format variables. Url link customisation will be documented at Wikipublisher.Customise "soon". By default, links to external web sites are shown as page footnotes. This can be changed to omit the footnotes, using the variable $UrlLinkTextFmt. For example, in local/config.php, after including the wikipublisher library, set:

    if ($format=='pdf') $UrlLinkTextFmt = '$LinkText';

PublishPDF