Avoka Blog: Adobe LiveCycle

April 7, 2009

Processes, Orchestrations, Services, and other confusing terminology

Filed under: Components, Designing Processes, LiveCycle, LiveCycle Architecture — htreisman @ 11:47 am

Introduction

There are a number of different terms used in Adobe LiveCycle Process Management, including process, orchestration, workflow, service, and component. These can be confusing.

Part of the reason for the proliferation of terminology is the history of the Process Management product and industry trends, and this blog entry tries to provide this perspective. This is my understanding of how these terms fit together, and should not be regarded as a definitive explanation.

If you don’t care about the historical perspective, skip to the end for a summary.

Workflow

The first version of Process Management was called “Adobe Workflow”. And the things you created were called “Workflows”. Even today, many people will generically refer to any combination of boxes and lines that you create in Workbench as a “workflow”.

However, the term workflow was used primarily in the 80’s and 90’s to refer to more traditional image storing and routing applications, and doesn’t really adequately describe the much more useful data-oriented and integration capabilities of LiveCycle. In the 2000’s, the terms BPM or Business Process Management started to become more widely used, not just as a technology, but also as a way of thinking about and improving your business. Various BPM disciplines evolved, including Six Sigma, Lean, and others. LiveCycle fits in much more closely with BPM than it does with the older imaging-oriented workflow systems, so…

Adobe Workflow became LiveCycle Process Management, and Workflow was replaced with …

Process

A Process is no different to a workflow, really, it’s just a term that better reflects the data-orientation and integration capabilities of the LiveCycle platform.

In LiveCycle, a process is almost always associated with a Form, and a series of human interactions that allow people to interact with the form and its data. There are almost always integration steps as well, such as pre-filling the form, writing to a database, adding a digital signature, sending emails, etc. These are also known as “long lived processes”, because they involve people interating with the form over a period of days or weeks.

Long lived process have a unique process-id, have every step recorded and audited, can be re-tried if any step fails, and can be viewed and managed through the Adminui interface.

Orchestration

In LiveCycle ES, we started seeing an increasing number of processes that are short-lived, and don’t include human steps. An example is the Default Render Process in LiveCycle ES. These processes used to be written as Java code in LiveCycle 7 – in ES, they have the advantage that being processes, we end-users can see and modify them very easily. It’s really just visual drag-and-drop programming. You can also easily build your own orchestrations to do something useful – for example, you may want to grab some data from an XML file, populate a form with it, encrypt the form with a password, and email the result to someone. There are no human steps in that, just a series of operation that need to be performed as quickly as possible. You could do this using the API’s, but implementing this as an orchration is quicker, more reliable, and much easier to change.

These short-lived or “straight-thru” processes do not store an audit trail, run as fast as possible (very close to native Java speeds), and do not have a unique process-id that identifies them. They cannot be managed through Adminui, and if they fail, they simply throw an exception back to the caller, rather than being able to be re-tried thru Adminui.

Someone coined the term “Orchestration” to refer to this type of process, and the name has stuck. I’m not sure where the use of this term originated, but it’s used in several other standards BPM standards and products.

Notes:

  • Adobe do not officially use this term in their documentation (see http://blogs.adobe.com/livecycledocs/2009/02/the_livecycle_terminology_secr.html) but I think it’s a helpful term that differentiates a process as being short-lived.
  • You can still use a long-lived process for something that doesn’t involve humans. We often use long-lived-processes for processes that geneally run very quickly, because of the advantages of having each step audited, and for the ability to re-try a step if anything goes wrong. I would still call this an orchestration, because it doesn’t involve any human steps.

Here is a screenshot from WorkBench. This shows a process, its implementation, and its properties. This is a short-lived process, so I would call it an orchestration.

process

Component

A LiveCycle Component is basically a bunch of Java code, packaged up into a jar file, and deployed into the LiveCycle server.

LiveCycle Components are brilliant. I love ’em. They are my insurance policy. I know that there is no problem that I can’t solve in the process world, because if i get stuck, I can write Java code to implement it. Then I can turn it into a LiveCycle component, drop it into LiveCycle, and bingo, I can use it my processes. And the real beauty of it is that anyone else can use my component too, without needing to understand programming. In LiveCycle, Adobe have created the best implementation of a server-side component model that I’ve ever seen – it’s a bit like those Visual Basic ocx controls that you can buy to build your GUI – but on the server.

The concept of LiveCycle server-side components came from a predecessor of Process Management, and were originally called QPACs – Quick (QLink for the real old-timers) Process Action Components.

Each component defines one or more services (see below). Although in reality, most components only define a single service, so components and services are somewhat interchangable.

A service can define one or more operations. An operation is the thing that actually does the work – like sending an email, encrypting a PDF, or writing a file to disk. Services group common operations together – so the File Utilities component contains a FileUtilsService, which in turn contains operations to read, write, create and delete files. Operations each define a number of input and output parameters.

For Java programmers:

  • Component == .jar file
  • Service == Class
  • Operation == method

Here is a screenshot of a Component within LiveCycle.

components

Avoka has been building components to solve real business problems with LiveCycle for many years, and we turn most of our components into re-usable components that anyone can use. We have a large library of components that solve most of the problems we’ve encountered over the last 5 years. So most of the time, you don’t even have to learn how to build components – if it’s not already in the Adobe “box”, then check out our website – there’s a good chance we may already have built it for you.

For more information, see: http://www.avoka.com/avoka/escomponents.shtml, or email info@avoka.com

Service

Another concept that was introduced in LiveCycle ES was that of a “Service Bus” or “Service Oriented Architecture”. LiveCycle is internally implemented as a service bus. In simple terms, any bit of functionality within the LiveCycle server can be exposed to the outside world as a “Service”, and can be invoked in a number of different and useful ways. Once you’ve defined a service, you can define one or more EndPoints for that service. So for example, I might have a service that encrypts a PDF – I could add a Web Service Endpoint that allows me to invoke that service using Web Services. And I could also add a Watch Folder endpoint that allows me to invoke that service by dropping a file into a folder.

Now things get a little confusing. Services can be implemented as either some Java code, or as a LiveCycle process.

  • If you need to write Java code, you create a component, deploy that, and you get a service.
  • Or you can do it by creating an orchestration (or process), you define your orchestration, activate it, and you get a service.
  • Either way, you get a service. The choice of whether to use an orchestration or Java code depends on the complexity of the service you’re trying to create, and whether it can be created by joining together (or orchestrating) a number of other existing services.

The LiveCycle process engine doesn’t really care whether your service is implemented in Java or as an orchestration, you invoke and manipulate it in exactly the same way. You can either call a service externally (by defining an endpoint), or you can invoke one service from another, simply by dragging the service into process. This is a very neat way to handle things – why should you care how something is implemented, you should be able to use it the same way.

Note: An orchestration only ever defines a single operation, always called “invoke”. Whereas a component can define multiple operations.

This screenshot shows two services, one implemented as a component, and the other as an orchestration.

services1

Summary

  • Process: Something that you build in Workbench, and looks like a flowchart. Can be user-oriented/long-lived, or straight-thru/short-lived.
  • Orchestration: An unofficial but commonly-used term for a straight-thru/short-lived process.
  • Workflow: An older term for a process.
  • Component: A jar file containing some code to do something “useful”, that you deploy to the LiveCycle server. A component contains services, which in turn contain operations. All the Adobe services are built as components, as are all the Avoka add-on services.
  • Service: Something that lives in the LiveCycle server that provides some sort of “service”. Can be implemented either as Java code (as a component) or as a process/orchestration.

March 2, 2009

Which “bit” of LiveCycle do I need?

Filed under: Components, LiveCycle, LiveCycle Architecture — htreisman @ 7:45 pm

Overview

LiveCycle has a lot of components, somewhere around 15 last time I counted. Some run on the server, some run on the client, some you cannot actually buy but are bundled with others – it’s really confusing, even for those of us who specialize in LiveCycle. But if you’re new to LiveCycle, we don’t blame you for feeling a little lost.

This blog entry attempts to match up your requirements to the various bits of LiveCycle that you may need.

I want to…

Put a “print and fill” PDF form on my site

What you need: Acrobat.

You don’t need LiveCycle, you just need Acrobat. Create your form using whatever tools you like (InDesign, FrameMaker, Word, even Excel) and convert it to a PDF using Acrobat. Your users can download it, print it out, and use a pen to fill it out.

What else you get:

  • You get the joy of having to open a whole lot of envelopes, sort the contents, and give the illegible hand-writing to someone to do the data entry into your systems.
  • You get the added joy of having incomplete or invalid information in the forms, and having to get back in touch with the user to gather the missing and/or correct information.
  • Sigh…

Put a PDF SmartForm on my site, and allow users to fill it out electronically and print it out

What you need: LiveCycle Designer.

Use LiveCycle Designer to create the SmartForm. If your SmartForm is pretty straight forward, you may be able to do this yourself. Alternately, you may need to attend a Designer Training Course, or enlist the help of an Adobe Partner. <shameless-plug>www.avoka.com </shameless-plug>. LiveCycle Designer is a very powerful tool, and you can do amazing things with it – but if you don’t have a programming background, you will quickly get lost in the more complex aspects of the tool.

What else you get:

  • You still need to have someone read the printed form and enter the data into your backend systems. However, it’s much easier to read a printed form than a hand-written one.
  • OCR (Optical Character Recognition) is much more reliable and accurate with typed forms than hand-written ones.
  • SmartForms can really make it easier for your end users by providing in-line help, can automatically hide those sections of the form that aren’t relevant, auto-complete, auto-fill, and provide other assistance to your users when filling in forms.
  • SmartForms can ensure that the data you receive is complete and error free, by identifying mandatory fields and validations. Forms won’t print until all mandatory fields and validations pass.

Allow a user to submit a SmartForm back to me via email

What you need: LiveCycle Designer

Use LiveCycle Designer to create the form, and add a “Submit by Email” button to your form.

Note: You will receive the form data as XML. You will then need to use Acrobat to re-inject this XML data back into a blank copy of your form. If you want to avoid this manual process, and allow your users to submit the form directly as PDF, you will need Reader Extensions for your SmartForm or Acrobat for your users (see below).

Allow a user to submit a SmartForm back to me via the web (i.e. http or https) avoiding manual data re-keying

What you need: LiveCycle Designer, some server infrastructure

Just use LiveCycle Designer to create the form, and add a “Submit” button to your form.

Notes:

  • You will need to create a Java servlet or ASP.net server or PHP server or some other technology to actually receive or process the incoming submission. This means that you will need to get programmers involved – they can use the tools of their choice – it’s just the usual web request “thing”.
  • Alternately, you can use a LiveCycle server to process the incoming http request. You will then be able to process the incoming request in a completely graphical “workflow” environment without any coding. You will need to acquire a copy of:
  • LiveCycle Foundation (at a minimum). You cannot buy Foundation, it is bundled with other LiveCycle services, so you will need to pick the most appropriate one. Contact info@avoka.com for advice.
  • Avoka’s Process Invoker. This is the “bridge” between the incoming web request and the LiveCycle engine. http://www.avoka.com/avoka/addons.shtml#invoker
  • If you want to turn the submitted XML back into a PDF, you’ll need either LiveCycle Forms or LiveCycle Process Management.

Allow a user to save a partially completed form offline or call a web service from within a Form or Submit or Send a form as PDF rather than XML

What you need: LiveCycle Designer and LiveCycle Reader Extensions for your form, or Adobe Acrobat for each of your users.

Using the free Adobe Reader, your users can save a copy of your blank SmartForm to their machines. But once they start entering data, they can’t save that partially completed form (they can only print it or submit it). There are also a number of additional capabilities that can only be performed in Acrobat (and not in Reader), including:

  • Invoking a web service from a form (for example, to convert a dollar amount to some other currency)
  • Commenting on a form
  • Digitally signing a form
  • Submitting or sending a form as PDF (a normal submission just sends the XML data contained in the form, but not the form itself)

If you want to allow your users to save a partially completed form, or perform one of these other actions, then you can purchase “Reader Extensions” for your form. By applying Reader Extensions to a particular form, this temporarily extends the capabilities of Reader to allow these features for this particular form. You can purchase Adobe Reader Extensions from Adobe Enterprise Partners.

Alternately, you can purchase a copy of Acrobat for each of your users – this may be more cost effective if you have a large number of forms and a small number of users.

Alternately, Adobe Acrobat provides a “cut-down” version of Reader Extensions built it. It only provides some of the above features, and is limited to a small number of forms and small number of end-users. Contact your Adobe Enterprise Partner to find out whether you can use this feature for your scenario.

Pre-fill a form with information that I already know about the user (or some other information)

What you need: LiveCycle Forms or LiveCycle Process Management

Your user has already logged into your site, so you know who they are and other information about them. When they open a form, it’s “polite” to pre-fill the PDF with information you already know about them, to a) avoid them having to re-key it b) reduce errors in their typing. LiveCycle Forms is a server product that that allow you to inject data into any LiveCycle Form prior to serving it up to your users. LiveCycle Process Management is actually a tool for automating human-oriented activities within your organization, but it also includes a light-weight version of the service that allows you to inject data into a form. Contact your Adobe Enterprise Partner for more information on which of these two services is best for your needs.

What else you get:

  • Common and Foundation

Provide a user with a printable record or receipt of their interaction with my site

What you need: LiveCycle Output

Your user has already performed some sort of interaction with your site (bought something, filled in a html or PDF form, made a booking, or whatever), and you want to present them with a PDF that is a permanent and printable record of that interaction. Build a LiveCycle SmartForm, and use LiveCycle Output to turn that form into a “flattened” PDF form. It’s now a regular PDF document, without the ability to make any changes to the values of any fields.

What else you get:

  • Common and Foundation (see below)
  • LiveCycle Output also includes facilities for printing PDF forms directly from the server to a network printer.
  • LiveCycle Output also allows you to create documents of record in an ISO archiving format.

Digitally sign my outbound documents to give my users a sense of security that it did come from me and hasn’t been changed

What you need: LiveCycle Digital Signatures

With Livecycle Digital Signatures, you can apply a digital signature to a PDF you’ve created with LiveCycle Designer, or any other PDF document. This can be signed with your corporate (or personal) digital signature. The signing process is automated on the LiveCycle server, and can be invoked programmatically from existing applications, or can be easily incorporated into human processes.

What else you get:

  • By signing a document, you make it tamper-proof – if anyone attempts (either maliciously or accidentally) to modify the document in any way, it will invalidate the signature.
  • Common and Foundation (see below).

Enter into binding agreements with my customers or suppliers without paper signatures

What you need: LiveCycle Digital Signatures and Reader Extensions

Create a PDF SmartForm with a digital signature field. Post this form on your web site. When users have completed the form, they can use their digital signature to sign the form, and submit it back to you as a signed PDF. You can use the Digital Signatures service on the LiveCycle server to authenticate the signature, obtain information about the person signing, and validate that the document has not been changed since it was signed.

You can use the signature for non-repudiation – in other words, if the person who signed the document says that it wasn’t them, you can show them their digital signature, and since only they can sign things with their digital signature, it had to be that person.

Note: You will need to have a closed user group where you can roll out electronic signatures. For electronic signatures to be valuable, they have to be issued by a certification authority who make you prove who you are, and will attest that you are who you say you are. (You can create self-signed certificates quite easily, but there’s nothing from stopping me from creating one that says I’m Warren Buffet. A certification authority will require proof that I am in fact Warren Buffet before issueing me a signature in that name. Some countries and organizations are starting to roll out digital certificates more broadly.)

What else you get:

  • By signing a document, your users make it tamper-proof – if anyone attempts (either maliciously or accidentally) to modify the document in any way, it will invalidate the signature. That way you can prove that it hasn’t been tampered with.
  • You can also add a digital signature field to any PDF document on the server (not just SmartForms), and send that to your users for signature.
  • You also get services for encrypting a PDF with a password or certificate.
  • Common and Foundation (see below).

Capture data from a printed PDF Smartform with 100% reliability and without Optical Character Recognition or manual data entry.

What you need: LiveCycle Barcoded Forms

Create a PDF SmartForm with a 2-D barcode field on it. The barcode can store up to about 512 characters of information, and can be set up to automatically capture information from the SmartForm fields as the user types into the form. When the form is printed, the barcode contains all the information entered into the form. Instead of OCR-ing the text in the form, simply scan the barcode and get all the data. 100% accuracy – you get all the data in one scan.

Turn Office documents into PDFs on the server.

What you need: LiveCycle PDF Generator

Pass any common office document (Word, Excel, Powerpoint, Open Office, many picture formats and other document formats) to PDF Generator, and it will turn it into a PDF. The recipient will not need to have the original application in order to read the document. Can be easily integrated into manual, ad-hoc processes, or invoked from other appliations in order to automate backend processes.

Add Reader Extensions, and cater for ad-hoc commenting and review processes using the capabilities of Adobe Reader.

What else you get:

  • You can also do a lot of other types of translations, such as images to PDF, PDF to images, etc.

Store any documents created using LiveCycle (or any other document) in a Document Management System

What you need: LiveCycle Content Services or ECM (Enterprise Content Management) Connectors

You may want to store your documents in a document repository for archiving, or so that users can find all document relating to a particular subject or case or client or whatever. Adobe provide ECM (Enterprise Content Management) Connectors that allow you to store, retrieve and manage documents in many different enterprise document repositories. If you don’t already have a document repository, Adobe provide Document Services, a light-weight but powerful and scalable document management system that is fully integrated with the LiveCycle system.

Content services contains it’s own user interface for browsing, searching, editing, and uploading documents called ContentSpace.

Create approval or back-office processes within my organization

What you need: LiveCycle Process Management

LiveCycle Process Management is a state of the art human-oriented Business Process Management System (BPMS). Processes are generally invoked when someone submits a form (although there are lots of other ways of invoking a process). You can then route a form to a manager or process worker for approval or action. You can also augment the data in the form by fetching additional data from other internal systems (e.g. databases), save form data out to internal systems (e.g. file system, document management systems), send and receive emails, add attachments, send reminders, automatically escalate or deadline tasks to ensure Service Level Agreements, and much more.

What else you get:

  • Workspace: An end-user portal where you can initiate new processes, or view your inbox to find tasks that have been allocated to you or a group you belong to.
  • Business Activity Monitoring: Dashboards and reports that allow you to view workloads and throughputs, bottlenecks in your process and much more.
  • By combining with other LiveCycle services, you can archive documents, merge documents together, send emails, etc.

Host documents on a public-facing site, with pre-population, save-online, receipts, branding, online payment processing, versioning

What you need: Avoka Form Center

Adobe LiveCycle provides all the core services to create, host, pre-populate and process SmartForms. However, if you actually want to create a portal for doing all these things, you have to build it yourself in Java or .Net or some other technology – it’s not provided “in the box”.

Avoka Form Center is built upon Adobe LiveCycle. It provides a rich form-hosting portal that allows public or occasional internal users to:

  • Locate forms
  • Fill them in
  • Save partically completed forms in a drafts location online
  • Pre-fill forms based on configurable “profile” information
  • Brand the same SmartForm differently depending on the origin of the request, or the user login details
  • Automate payment processing for forms that incur a cost
  • Provide a history of all submitted forms
  • Provide an automatic receipt on completion

Form Center also provides an extensive administrative module, that allows forms to be uploaded, configured, versioned, branded, and much more.

http://www.avoka.com/avoka_formcenter/formcenter.shtml

What are “Common” and “Foundation”?

Common and Foundation are a set of lower level services that are bundled along with other LiveCycle services. You cannot buy these on their own. Foundation is always included with every LiveCycle product. The Common services that are bundled vary from product to product. Please contact info@avoka.com for more details if you’re unsure.

  • Common includes services for combining multiple PDFs into one, adding watermarks, adding table of contents, headers, and footers, extracting and injecting data and meta-data, encryption, and more.
  • Foundation gives you fantastic integration capabilities with databases, email, file system, messaging and directory services, XML manipulation, and more. This is stuff that the EAI (Enterprise Application Integration) vendors charge you big bucks for, and Adobe gives you for free (with other products).
  • The orchestration engine underlying the whole of LiveCycle provides a graphical user interface for defining your “workflows” – no coding required.
  • The orchestration engine is inherently extensible – we’ve never found something that a client wanted it to do that we couldn’t make it do.

February 9, 2009

Using the Avoka XQuery Service Component

Filed under: Components, Designing Processes — Jason Hendry @ 8:46 am

Livecycle has some useful Service components for manipulating files, database rows and XML documents. And while XSLT is a powerful and useful language, it can sometimes be a little unwieldy to express simple logical problems.

I recently had to perform some simple logical programming on the server side to select one of three text fields on a form. The text fields were comments from users and the last user to touch the form before exiting the flow had their comments inserted into an email back to the form originator. There are a multitude of different ways of handling this problem on the form instead of on the server, however, this problem came to me as a system maintainer and there were constraints I had to observe.

Avoka’s XQuery component makes it possible to construct strings and XML fragments using the XQuery language. XQuery, as you may know, has a syntactic structure called FLWOR or

For
Let
Where
Order
Return

This is typified by the example given on W3Schools using the Books XML data sample:

for $x in doc("books.xml")/bookstore/book
where $x/price>30
order by $x/title
return $x/title

*Note that the Let is implied in $x in doc(…)/bookstore/book

But what I needed was not an iterative solution (at least, not one that required a for loop), but a way to simply express the conditional logic to select the first non-empty XML text node.

To test the hierarchy of user’s for the last commenter, I could have used a number of Decision and Set components, but why take 3 steps when you can take 1?

Alternately, I could have implemented similar logic using the Livecycle script component and while it would have performed in a single workflow step, the script component can be unwieldy when manipulating form values. Do you really want to …

patExecContext.getProcessDataStringValue("/process_data/status");

… just to test a couple of values?

Step 1: Workflow

There are two possible types of invocation on the XQuery Service Component, Single Document and Merge Document.  The Merge Document invocation is intended to be more powerful so in this example we’ll be considering the Single Document invocation of the XQuery component.

I inserted the XQuery component (Last Commenter) into the workflow, just before the email step to notify the originator of the last user’s comments.   This is where we’ll test the form data variable for the comment fields, and then copy them into another process variable to be inserted in the email.

Inserting the XQuery service object into the workflow

Inserting the XQuery service object into the workflow

Step 2: Service Configuration Parameters

The XQuery object is fairly simple to setup, only requiring an XQuery statement and an Output location. In the example here, I’m using a variable, emailCreditComments, to capture the comment tex which I’ll re-use in the following email workflow step.

Configuring the Avoka XQuery Service Component

Configuring the Avoka XQuery Service Component

Step 3: XQuery Statement

First, I chose a location in my process variables to act as the root of my XQuery statement. This is a parameter of the statement, rather than the XQuery configuration and acts to reduce the size of the document the XQuery statement operates on.  Since I’m looking for comment fields, I set the location deep into my XFA form variable structure:

/process_data/form/object/data/xdp/datasets/data/DocumentRoot

In this example, I’m using the root of the XFA data passed into the process as a form variable.

Next, as a matter of style, I set the form comments text into re-usable variables that I can test.

let $sp := //FormData/SeniorPartner/Comments/text()

Then, I constructed a logic statement that allowed for graceful regression without forcing the flow. In most functional programming languages, this style of cascading if() statement is to be avoided since it can become a real headache for maintainers. However, the nature of service components in Livecycle means that we can consider (and re-use!) pieces of functionality independently.

let $sp := //FormData/SeniorPartner/Comments/text()
let $hod := //FormData/HeadOfDepartment/Comments/text()
let $mgr := //FormData/Manager/Comments/text()
return if (string-length($sp) = 0)
	then if(string-length($hod) = 0)
		then $mgr
	else $hod
else $sp

As you can see the statement itself is quite simple, cascading from each if() statement, testing for validity in the order of the user hierarchy (Senior Partner, Head of Department, Manager). Validity in this case is a test for string length, such that any test that fails indicates we have a non-empty comment text node.  The first test to fail gets it’s comment text returned via the else clause and inserted into the configured output location.

XQuery Service Configuration Statement
Lastly, since the result of the XQuery statement will be inserted into an email, rather than re-used as an XML data set, we tell the XQuery service component to omit the XML processing instruction. Other options include the ability to ‘pretty print’ the output and to re-use namespaces from the source process definition.

Summary

This example demonstrates a simple, if somewhat unusual use of the XQuery language implemented using the Avoka XQuery Service Component.

References

http://manly.avoka.com/confluence/display/Public/XQuery+DSC

http://www.w3schools.com/xquery/default.asp

Blog at WordPress.com.