Using MorganaXProc’s Command Line Interface

In this document we will discuss how to use MorganaXProc's command line interface, which is an alternative to the graphical user interface. If you do not want to use the command line interface, you can skip this document and go back to the user's guide overview.
It is supposed that you are familiar with the basic terms of XProc and xml, such as pipeline, input and output ports, options, parameters and so on.
Overview:

In a hurry? There is also a a quick reference guide for using the command line.

The command line interface (CLI) for MorganaXProc is an easy to use tool to run your XProc pipelines, to connect input ports of your pipeline with xml documents on your computer or the web, to set options declared in the pipeline and to save the xml documents on the output ports of your pipeline to any place you want. Lastly you can also configurate MorganaXProc by using the command line interface.

Which settings of the command line interface you will actually use, depends on the XProc pipeline you want to run. To explain the command line interface we will use the XProc pipeline welcome.xpl which you already know from the installation procedure. This pipeline has one input port named "source" (being a "sequence input port") and one option named "user". So let us answer the basic questions:

How to call the command line interface?

As you have already seen, MorganaXProc's command line interface is activated by typing "MorganaXProc" (on Windows) or "sh MorganaXProc" (on MacOS) with the path set to the folder named "MorganaXProc". So the way to call the command line interface is this:

  1. Open the command line interpreter application ("Command Prompt" for Windows, "Terminal" for MacOS).
  2. Navigate into the folder "MorganaXProc", actually into the folder (with any name you like) which contains MorganaXProc.jar.
  3. Type "MorganaXProc -help" (on Windows) or "sh MorganaXProc -help" (on MacOS). If you are familiar with the standard Java procedure, you can of course also type "java -jar MorganaXProc.jar".

If you now hit <return> MorganaXProc's command line interface will answer this:

--------------------------------------------------
This is MorganaXProc 1.0.6
Copyright 2011-2017 by xml-project Achim Berndzen
Usage:
pipelineURI {[‑input:portname=uri] | [‑output:portname=uri] | [‑option:optionname=value] | [‑config:uri]} or

-testsuite=path-or-uri [‑output=filepath] [‑first=test-filename] [‑last=test-filename] [‑asciidoc=[yes/no]]

--------------------------------------------------

If you forgot how to use the command line interface and do not want to check back to this page, MorganaXProc will put out this as a reminder of how to use it. Let us look at this in detail.

How to name the pipeline I want to run?

The first thing you will want to do is to name the XProc pipeline you want MorganaXProc to run. The input for MorganaXProc's command line interface consists of one or more elements, each of them starting with a <space>. The first element for the command line interface is always the name of the XProc pipeline you want MorganaXProc to run. You can give the name either by typing the full uri of the pipeline or by giving a relative path to the pipeline. If the uri is relative, MorganaXProc's command line interface will try to identify the XProc pipeline to run in two ways:

  • You can set the home folder for your pipelines by using the configuration property PipelineFolderURI. Having done that, MorganaXProc will use the uri given with this property to identity the pipeline to run any time a relative uri is found. We will discuss the configurations later. For now it is just important to keep in mind the possibility to set a home folder for your pipelines.
  • If no setting for PipelineFolderURI is found, MorganaXProc's command line interface will use the current working directory to find the pipeline for the relative name. This means it will use the folder, you navigated to before calling MorganaXProc. You can see the name of this folder by looking on the start of the command line prompt.

Lets give it a try: Type "MorganaXProc pipelines/welcome.xpl" (Windows) or "sh MorganaXProc pipelines/welcome.xpl" (MacOS). You have done this before while testing the correct installation, so it should work. The result should look like this:

--------------------------------------------------
This is MorganaXProc 1.0.6
Copyright 2011-2017 by xml-project Achim Berndzen
Using Apache's HttpClient(TM) (c) 2005-2014 The Apache Foundation
NOTICE: PortSerializer supports method 'mox:binary'
--------------------------------------------------

<?xml version="1.0" encoding="UTF-8"?>
<greeting>Standard greetings from MorganaXProc to the user.</greeting>

--------------------------------------------------
Bye from MorganaXProc 1.0.6

Now you should know, what happens here: You called MorganaXProc while the relative pipeline name "pipelines/welcome.xpl". The command line interfaces now look for the configuration property PipelineFolderURI which names your pipeline home folder. Having this property not set yet, the command line interface takes the current working directory as base for finding the pipeline "pipelines/welcome.xpl". This happens to be the folder "MorganaXProc", so the pipeline "welcome.xpl" is taken from the folder "pipelines" in the folder "MorganaXProc".

What can go wrong here?

  • You might get an error message like "An error occurred: /pipelines/welcome.xpl (No such file or directory) [java.io.FileNotFoundException]" from the command line interface. This happens because the pipeline name you typed starts with an "/". An uri starting with "/" is always taken as a path starting from the root directory of your system. But the pipeline you want to run is not in the folder "pipelines" which is a folder in your root directory, but is a folder in "MorganaXProc" which is a folder somewhere in your file system. So: Just omit "/" and everything will work as expected.
    Of course you can also give the name of your pipeline by starting the path from your file system's root directory. In this case you have to start the path with an "/" and then give the full path to the folder "MorganaXProc" in your file system and then end up with "/pipelines/welcome.xpl".
  • Another common error message might look like this: "An error occurred: /Users/me/document/MorganaXProc/pipelines/welcome1.xpl (No such file or directory) [java.io.FileNotFoundException]". As you see, you now get not just the path you typed in (which was the case in our first error message), but you get an expanded path showing you, where MorganaXProc tried to find the pipeline. However, the result is the same: The pipeline was not found. This time the reason for this error message is fairly simple: We misspelled the path for the pipeline by typing "pipeline/welcome1.xpl" which names a non existing pipeline.
    Please keep in mind, that you are expected to write (a segment of) an uri. That means you will have to escape special characters allowed in your file system but not in an uri. The most prominent character allowed in most file system is a blank in a file name. This is as is generally known not allowed in an uri, so you have to escape the blank with an "%20": To run an XProc pipeline from a file named "welcome a.xpl" you will have to write "welcome%20a.xpl".
  • A third type of error message might occur from MorganaXProc's security manager and looks like this: "An error occurred: Security exception: RUN_PIPELINE is forbidden for 'https://www.xml-project.com/files/pipelines/welcome.xpl' [java.io.IOException]". You get this type of error if you try to run a pipeline from a source that is forbidden by MorganaXProc's security management. Here it is "https://www.xml-project.com/files/pipelines/welcome.xpl" because the default security settings forbid running pipelines from other sources than your file system. We will see later, why this is forbidden by default and how you can modify MorganaXProc's security settings.
  • The fourth type of error that might occur here, is related to the content of the pipeline to are trying to run: You might get some error starting with "An error occurred: Document 'welcome.xpl' is not valid". You get this type of error if MorganaXProc is not able to build an xml document from the file you named, because the file does not contain a valid xml document. Look at the text of the error message to get a hint at what is wrong with your xml document.
  • Finally there may be errors occurring during compiling the XProc pipeline or from running the pipeline. Those error messages are defined with the language itself. See the List of Error Codes for an explanation of this error messages.

How to connect an input port?

Now you know how to run a pipeline from the command line interface. This is pretty easy for pipelines that do not have an input port. But most XProc pipelines will have at least one input port, because XProc is about getting xml documents on an input port, doing something with this document and then produce another xml document on an output port. So you might ask yourself, how to connect xml documents to the input ports of your XProc pipeline.

To do this you have to put an "-input:" element into your command line. Let us use an example again: The XProc pipeline "welcome.xpl" has an input port named "source" (which is a "sequence input port", that is why we could run the pipeline without connecting the port.) And we want to connect this port to a document in the file "greetings.xml" which is located in the folder named "pipelines" in your "MorganaXProc"-folder.

Type:

  • Windows: MorganaXProc pipelines/welcome.xpl -input:source=pipelines/greetings.xml
  • MacOS: sh MorganaXProc pipelines/welcome.xpl -input:source=pipelines/greetings.xml

and you will get:

<?xml version="1.0" encoding="UTF-8"?>
<greeting>Special greetings from MorganaXProc to the user.</greeting>

As you see, the content of the greeting element has changed. The text comes from the document on the input port. So: Connecting an xml document with an input port of your pipeline is simple: Just write the "-input:" element anywhere after the pipeline name into your command line. (Remember: The name of the pipeline you want to run always has to be the first element.) The "-input:" element is followed by the name of the port you would like to bind and by "=". Then add the uri of the xml document your would like to bind and hit <return>. Done!

As you have seen, binding a document to a port is pretty natural if you know how a run a pipeline. In fact the same mechanism for obtaining the document applies here: If you give an absolute uri the input for the named port will come from the ressource with this uri. If you type in a relative uri, this is made absolute by MorganaXProc. But this time it will not use the settings for the property PipelineFolderURI but from another property called DocumentBaseURI. If this property is not set (as in MorganaXProc's default configuration), the command line interface will again use the current working directory as a base for making the relative uri absolute.

Now: As we have said, input port "source" of pipeline "welcome.xpl" is a "sequence input port" which means, it can have zero, one or more documents. We have already seen how to connect zero documents to this port (just leave out any element "-input:source") and how to connect one document. How to connect more documents? Well, you already know the mechanism. Just add another element "-input:source=" to your command line and add any (absolute or relative) uri you like. By doing this, you add a second document to input port "source" and the same mechanism applies for any further document. (But do not forget: Elements in the command line are separated by blanks! There must be a blank before any "-input".) The order in which the documents appear on the input port's sequence is of course the same order in which you bind the documents in your command line.

What about error messages?

Well, some things may go wrong, while connecting an xml document to a pipeline's input port. The most obvious ones are the same as while naming the pipeline to run (discussed above). You may have written a relative path as an absolute one (starting with "/"), you may have misspelled the uri and thereby named a none existing ressource. You might have not escaped special characters in your file name not allowed in an uri. Finally, you may have tried to access a ressource forbidden by MorganaXProc's security management.

What else can go wrong:

  • The most common error will be, trying to connect to a non declared port (mostly by misspelling the port's name). In this case you will get an error message in MorganaXProc's output telling you, which port name was not declared: "XProcPipeline: Port 'source1' is bound but not declared in pipeline 'welcome'." (Hint: You only get this error message, if in MorganaXProc's configuration the property StrictExternalBinding is set to true. This is the default setting. If you set this to "false", MorganaXProc will ignore bindings for undefined ports. The error you will get then depends on your pipeline. In some cases you will even get no error at all, but the output of the pipeline will not be the one you have expected.)
  • Another common error will be to forget the binding for a port that is not a "sequence input port" and that therefore needs a binding. In this case you will get an error message with XProc's error code XD0006, telling you, which port's binding is missing. You will get the error code as well, if you are trying to bind more than one document to a port, that is not a "sequence input port".
  • Lastly you may get an error message telling you, that you missed the correct syntax of the command line interface. This will probably result from a missing "=" between port name and document uri.
  • If anything else goes wrong: Please keep in mind, that MorganaXProc's command line interface is pretty relaxed about misspellings of an element name: It will simply ignore any element it does not know. So if you type "-inputx:..." instead of "-input:...", the command line interface will ignore the unknown element name "-inputx" and no xml document will be bound to the input port of your XProc pipeline. Sad, but true.

How to set an option?

Now, what about setting options in your XProc pipeline using MorganaXProc's command line interface? Well you might have guessed it! Use the "-option:" element anywhere after the pipeline's uri. Add the name of the option as declared in your pipeline, attach "=" and then give the value you want to add. It's as simple as this. Well, not really! But let us pretend for a moment it is.

Type:

  • Windows: MorganaXProc pipelines/welcome.xpl -input:source=pipelines/greetings.xml -option:user=me
  • MacOS: sh MorganaXProc pipelines/welcome.xpl -input:source=pipelines/greetings.xml -option:user=me

You will get:

<?xml version="1.0" encoding="UTF-8"?>
<greeting>Special greetings from MorganaXProc to me.</greeting>

As you see, the result document of the pipeline has changed again, now giving special greetings to "me", or any other value you will set for option "user". Ok, where are the problems using the "-option:" element? The first one is pretty obvious if you recall that the elements in the command line interface are separated by a blank. What do we do, if we want the pipeline to give greetings to a person named by first name and family name. The first try may be to add "-option:user=John Smith" to the command line. See what happens, if you do this. The greeting will go out to any person named "John", but the family name is lost. This is because the command line interface separates elements by a blank and so the blank between "John" and "Smith" is taken as an indication for a new, yet unknown element named "Smith". What do we use to come up with this problem? We will use quotes! Just put quotes around the value you want to set and everything will be fine. So "-option:user='John Smith'" gives you the result you are looking for. Of course you can either use single or double quotes, but take care to use them in pairs only.

Is that all to know about setting values for options? No, I'm sorry. As you might recall from the XProc definition, an option name can be a qualified name (a QName), that is a name with a namespace uri attached. Such a definition might look like:

<p:option name="mox:user" xmlns:mox="http://www.xml-project.com" />

You can set the value of an option declared as a QName by using the so called Clark notation for the option name, that is "{uri}local". So if you want to set a value for the option "mox:user" in your pipeline you have to write "-option:{http://www.xml-project.com}user='value'".

Are we done now? Yes. That is all you need to know about setting values for options in your pipeline. This time, the errors that might occur are obvious (misspelled names, missing values) and from what you learned about binding ports, you known how to deal with them.

One more thing: How to save the output(s) of a pipeline?

So far we have seen, that MorganaXProc writes the output of an XProc pipeline directly to your command line interpreter application. This is handy if you just want to see the results. But sometimes, probably even most of the times, you may want to save the output(s) of the pipeline so you can keep it and work with it. To do this, we will use another element for the command line, for obvious reasons named "-output:". As a general rule, MorganaXProc will write the documents on every output port of a pipeline to the command line interpreter application, unless there is an element "-output:" for this port.

How to use the "-output:" element? As you might have guessed you need to attach the name of the port you want to save to "-output:", then again we will use "=" to separate the name from the following uri. For example: "-output:result=file:///users/me/documents/result.xml". This will save the documents on the output port named "result" to a file with the given uri. If the uri is relative, it will be made absolute by using the current working directory. In our current environment, this means the result file will be stored in the save folder as "MorganaXProc.jar".

If your output port is a "sequence output port" and contains more than one document, the documents will be written to file sequentially in the order of appearance. Caution: This means that the result document produced might not be a valid xml document because it has as many "root elements" as there were documents on the result port.

Have to tried it yet? You should get an error message like "An error occurred: Security exception: STORE_RESOURCE is forbidden for..." This is because of MorganaXProc's security settings. By default, no pipeline is allowed to store anything on your system. You have to change to security settings to be able to store any results on your disk. We leave this for now and will discuss this point when talking about security management.

Since MorganaXProc 1.0.6 you can use command switch -unsafe to store pipeline results via command line processor (and only here) without invocation of the security system. Any resource previously associated with the given URI will be overwritten with the respective pipeline results, so please by careful.

Last question: How to set parameters?

The answer to this question is short but disappointing: You can not set parameters from the command line interface. Instead, you have to connect a full xml document to a parameter input port.

Now you know how to use MorganaXProc's command line interface, let us try to understand the configuration options.