Orbeon Forms Blog

Orbeon Forms has long supported exposing XPath variables with the xxforms:variable extension [1]:

<xxforms:variable name="answer" value="42"/>

So it’s great that XForms 2 now standardizes variables with the new xforms:var element! The above example becomes:

<xforms:var name="answer" value="42"/>

The working group went with var for concision (not to mention that the name will be familiar to JavaScript programmers!).[2]

With XForms 2, you can use variables[3]:

interleaved with controls
within actions

Immutability is a good thing in programming, and XForms variables were designed this to be immutable within a given context. In particular, you cannot set a variable value in XForms (or XSLT): the variable value updates “when needed” based on the expression in the value attribute, respectively when the control tree is refreshed, or when entering the scope of an action.[4]

You refer to a variable using the $ notation of XPath:

<xforms:output
    value="if ($answer = 42)
           then 'Answer to the Ultimate Question'
           else 'Wrong!'"/>

Variables are also very useful within actions:

<action iterate="books/book">
    <var name="pos"  value="position()"/>
    <var name="book" value="."/>
    <setvalue ref="$book/@id" value="$pos"/>
</action>

You can do this kind of things with plain XForms 1.1, but then you would have needed to use the less clear context() function, for example.

Orbeon Forms 4.0 milestone builds support the standard xforms:var, so feel free to move your code away from good old xxforms:variable!

See the documentation on Variables in Orbeon Forms. ↩
When Orbeon Forms introduced variables, it first used a select attribute to specify the variable value, partly inspired by XSLT. But select didn’t feel right in XForms, and of course using “value” is natural as in most programming languages, a variable has a name and a value! ↩
Orbeon Forms supports, in addition, placing variables directly under the xforms:model element. XForms 2 does not standardize this yet. ↩
This could have made a case for calling the element val instead of var, as the Scala programming language does for immutable variables. ↩

Today we released Orbeon Forms 4.0 M16 (Milestone 16). Like 4.0 M15 and the previous milestones this is not a final release.

In M16, we addressed the following issues:

Form Builder

FB: cut to clipboard fails if fr-form-binds id is missing (#579)
FB: tabbing out of field reaches delete icon (#580)
FB: don't run custom event handlers at design time (#583)
Fix crash when section template is non-existing (91cee9a750)
Section editor is in wrong place (#572)
On Firefox, in the section label editor, pressing enter does not close the editor (#585)
FB: cut/paste control with bind/@nodeset adds bind/@ref attribute (#588)

Form Runner

Improvements to fr:number

Add fr:number field to Controls form (#399)
fr:number: decimal formatting not working (#547)
fr:number: support blank thousands separator (#587)

Value of components in section templates are not saved (#596)
CE: DMV-14 PDF returns XML document (#253)
fr:section doesn't output grouping span elements
fr:grid: exclude +/- from tab navigation
Move forms to eXist

Controls form
Bookshelf form

Hide help and hints in PDF
Regression: With MySQL, Form Builder summary page doesn't show app/form names (#598)

XForms engine

Page crashes if xforms:output refers to not found resource (#582)
Document implemented XBL modes and related code updates (#394)
Support XBL "value" mode (#437)

Other

User can download page flow XML (#564)
Upgrading jQuery from 1.7.1 to 1.8.2
Standardize usages

xforms:var (#369)
ref instead of nodeset
xforms:property instead of xxforms:context
targetid instead of target on xforms:dispatch
xforms and xhtml namespaces (#368)
xf:trigger instead of fr:button

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.M16.201211200609-PE.zip
CE version: orbeon-4.0.0.M16.201211200623-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Today we released Orbeon Forms 4.0 M17 (Milestone 17). Like 4.0 M16 and the previous milestones this is not a final release.

Due to the Thanksgiving break, M17 is a rather small update, but it happens to contain two important new bits:

a new landing page for the Form Runner and Orbeon Forms examples (#165)
a new "wizard" mode which allows showing forms one section at a time

In addition, the following issues were addressed:

Form Builder

No default button set in some dialogs (#571)

Form Runner

fr:alert-dialog with static label crashes (#607)
fr:error-summary: improved recalculate and use xxforms-visited/unvisited (ab3ff7f890)

XForms engine

xxforms:value() function (1021d5eade)
New `focusable` mode for XBL components (#526)
TinyMCE with required content doesn't reliably show error (#301)
Change of visited state requires refresh (84a13e93c8)
New xxforms-visited/unvisited events (f6c02b4377)
Add deferred flag to xf:recalculate action (3339a5c2c2)

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.M17.201211280748-PE.zip
CE version: orbeon-4.0.0.M17.201211280802-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Data persistence for forms created with Form Builder is done through a REST API. You can implement that API, or use one of the provided implementations. Orbeon's implementation for relational databases, for now specifically MySQL and Oracle, always uses inserts instead of doing updates or deletes¹. For this, tables have a timestamp column, and when data is "updated", a new row is added with the current timestamp. So to retrieve data, we need to get the row matching a condition that has the latest timestamp.

Let's transpose this problem to an equivalent one on the emp table, to make it easier for you to run the queries below in your own database². On the emp table, a similar problem is to find for each job, the employee with the most recent hire date³. This can be done with nested queries:

select e1.* 
  from emp e1,
       (  select job, max(hiredate) hiredate 
            from emp 
        group by job ) e2
 where e1.job = e2.job
       and e1.hiredate = e2.hiredate ;

A bit complicated isn't it? This is where Oracle's dense_rankcomes in. This analytic function allows you to rank data. The following adds a rank column, and for each job, the employee with the latest hire date will get a rank of 1, the employee hire just before that a rank of 2, and so on.

select e.*,
    dense_rank() over (partition by job order by hiredate desc) rank
from emp e;

From here, we just need to keep only the employee with rank = 1 to get the expected result:

select * from
    (select e.*,
            dense_rank() over
            (partition by job order by hiredate desc) rank
       from emp e)
 where rank = 1 ;

¹ This allows auditing and allows an admin to retrieve older version of the data if necessary.
² In Oracle, this table is normally found in the scott schema, and you don't have it, or would like to import it in another database, you can download the SQL to create the table and add data.
³ For the purpose of this discussion, we'll assume there is only one employee for each job with the most recent hire date.

One of the cool new features of Orbeon Forms 4.0 is the wizard view.

By default with Form Runner all the form sections appear in the same page, on top of each other. If your form is large that means that you have to scroll to fill out the entire form.

With the wizard view, top-level sections instead appear in a navigation area to the left, and only a single top-level section is shown at any given time.

You can navigate between sections by clicking on a section title, or you can use the navigation arrows.

Errors on your form appear at the bottom as usual, and sections that contain errors are highlighted in red. If you click on an error you are taken directly to the section and control containing the error.

The wizard view is optional - you can use the regular view instead, and you can enable this view per form, per app, or globally with a property:

<property
as="xs:string"
name="oxf.fr.detail.view.appearance.*.*"
value="wizard"/>

We hope you enjoy this new feature!

Today we released Orbeon Forms 4.0 M18 (Milestone 18). Like 4.0 M17 and the previous milestones this is not a final release.

In this release, the following issues were addressed:

Form Builder

Form Builder should use logo specified in oxf.fr.default-logo.uri (#616)
Permissions dialog layout scrambled (#570)
Hover over button doesn't show proper button editor (#553)

Form Runner

FB summary page doesn't show app/form name for new forms with MySQL persistence layer (#611)
eXist persistence not to throw an error when the database is empty (#614)
Form Runner home page: message for no forms, include forms without permission (#94)
Don't highlight title on hover (#615)
Lots of CSS and cosmetic improvements
fr:number focus in/out loses formatting (#586)
Wizard: better navigation buttons

XForms engine

Server sets index of todo instead of list when re-adding first list (#591)
Non-visible dialog content should be non-relevant (#15)
Tooltip for a trigger hint in a dialog shows below the dialog (#620)
Add deferred flag to xf:rebuild/xf:revalidate actions
Move xforms.css to LESS
Refactoring of control bindings support

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.M18.201212050505-PE.zip
CE version: orbeon-4.0.0.M18.201212050521-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

There are times when you'd like to create a service, want the service to be available publicly, and want to do this quickly, maybe for testing or for a demo you're putting together. In those cases, Google Apps Script might just be the solution you're looking for.

As an example, let's create a service that tells us if a number it receives is even. The number will be passed as a request parameter, e.g. ...?number=42, and will provide an XML response, e.g. <result>true</result>. Then, we will call this service from a form created with Form Builder, to show, next to an input field, whether the typed number is even or odd. Let's start by creating and deploying the service:

To create a new script, load Google Drive, click on Create, and under More, choose Script. Click on Untitled project, and name it IsEven.

Edit your script, or in this case copy-pate the following code in the editor:

function doGet(request) {
    var result = '' + (request.parameter.number % 2 == 0) + '';
    return ContentService.createTextOutput(result)
        .setMimeType(ContentService.MimeType.XML);
}

To deploy your service, make sure it is first saved, then go to File | Manage Versions, click on Save New Version, and click OK to close the dialog. Go to Publish | Deploy as web app…, in Who has access to the app choose Anyone even anonymous, click Deploy. Copy and save somewhere the URL given to you in the following dialog: this is the URL of your script.
Test the service by pasting the URL in a new tab of your browser and adding ?parameter=42. The service should respond <result>true</result>.

Now let's call the service from a form we create with Form Builder:

In Form Builder, create a new form, create an input field, type a label and name it number, create and output field, type a label and name it even. Your form might look like:
Define an HTTP service, by clicking on Add under HTTP Service in the sidebar. Name the service is-even, in Resource URL paste the URL from step 3 above. Under Serialization, choose HTML form. In Request Body enter <number/>.
Define a action, by clicking on Add under Actions in the sidebar. Name it check-even, under React To choose Value Change, under Control choose number, in Service to call to choose is-even. In Set Service Request Values click the plus icon choose number and type /number, in Destination Control choose even and type /result.
Test the form by clicking on the Test button. Type 42 and hit enter: the output next to it should show true. Type 43 and hit enter: similarly the output should show false.

Congratulations, you just created a service and a form calling that service, and all this without leaving your web browser.

Today we released Orbeon Forms 4.0 M19 (Milestone 19). Like 4.0 M18 and the previous milestones this is not a final release.

In this release, the following issues were addressed:

Form Builder

Copy/paste of control loses translation (#625)
Enable XPath type annotations by default in Builder (#630)
When creating a new form, dialog shouldn't show a 'cancel' link (#590)
Form settings: textarea is not tall enough (#622)
Setting help to blank doesn't remove help (#577)
Custom XML: focus taken out of XML editor on click (#642)
Custom XML: dialog is too narrow, not high enough (#643)
Should not be able to set the focus on a readonly CodeMirror (#645)
Regression: Action editor: input/output values can't be set (#647)
403 error when testing form containing an XPath error (#627)
Wizard view by default for orbeon/controls
Make fields required in Action Editor

Form Runner

Required asterisk doesn't show in dialogs (#639)
fr:number is now documented (#546)
Add fr-grid-col-<x> classes on cols and fr-grid-<id> on table (a4bcef1766)
More files converted to LESS

XForms engine

Increase XPath cache size for very large forms (1eae552448)
Some performance improvements (4e9d661dba, 4c619bd5a6)
Calendar resource in Polish (da59e94de1)

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.M19.201212120146-PE.zip
CE version: orbeon-4.0.0.M19.201212120202-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Nowadays, almost everything uses Unicode, which supports a large (very large!) number of characters. Unicode assign a code (number) to each character, and in most cases this code is represented with:

With UTF-16 when in memory (e.g. in Java and Windows since NT);
With UTF-8 when sent over the wire.

Before Unicode, a single byte per character was used for Western languages, with the ISO Latin-1 encoding. This encoding was fine for most Western characters but didn't contain some useful characters, such as curved quotes, the euro sign, the trademark sign, plus many others. In their infinite wisdom, Microsoft decided to use some reserved codes of Latin-1 for those "useful" characters, creating the Windows 1252 encoding.

Unicode is based on Latin-1, and not Windows-1252, which means that the code for all those "useful characters" is higher than 255 in Unicode. The problem is that documents encoded with Windows-1252 are often incorrectly opened as Latin-1. The mistake is easy to do, as it works "in most cases". The error is so common, that the HTML5 spec says that a browser should parse document advertised as using Latin-1 as Windows-1252 (not trusting the advertised encoding!).

But should you incorrectly take that Windows-1252 encoded file as a Latin-1 encoded file, and pass along its content to a system where it is saved as Unicode, you might end up with control characters; if the text is sent back to the browser, still as Unicode, those control characters will show as squares, instead of the curved quotes, euro sign, or trademark signs you originally intended.

Luckily, there is a way to safely and automatically fix incorrectly encoded documents. This is done by changing the code for characters that exist in Windows-1252, but not Latin-1, to their valid Unicode code, using a simple conversion table. And Orbeon automatically does this for you. And of course, configuration properties allow you to disable this automatic conversion, or even to setup your own custom conversion.

Today we released the first Beta release of Orbeon Forms 4.0! Like 4.0 M19 and the previous milestone builds this is not a final release.

We have addressed lots of issues in this Beta release, covering Form Builder, Form Runner, Liferay support, and more. We also addressed a bunch of UI and cosmetic issues and increased the amount of testing we have done. With this, we have closed almost all the known bugs we wanted to address for 4.0.

Here is the overall list of changes since M19:

Form Builder

FB: control validation dialog is too tall and fieldset legends too big (#471)
Custom buttons are showing in Form Builder (#658)
Wizard does not show in test when testing a form from Form Builder (#624)
Unclear spacing for custom control properties (#383)
Message about XPath error when using $fr-roles in the builder (#354)
Fix "XXXXXXX" in model.xml (#632)
Better alignment in Validation Properties (#671)
Better labels in Edit Grid Details dialog (#672)
List of languages is hard to use (#685)
Wrong row deleted (#683)
Schema dialog: can't see all simple types (#692)
Validation dialog shows Double when Schema type is picked (#693)
Permissions not saved (#695)
FB Test fails with 403 (#702)

Form Runner

Review page: textarea uses monospace font (#631)
Add property for supplemental CSS (#617)
Schema generation fails when using eXist persistence layer (#666)
Section titles are 3 times taller (#669)
Accessible link in navbar is too tall (#656)
Review mode: repeat shows +/- icons again (#673)
Buttons touch the bottom of the viewport (#654)
Repeat: missing rowspan messes up table (#675)
User with only Create role can see forms on summary page (#696)
Excel import fails with 403 during validation (#701)

XForms engine

With Chrome, upload control alignment is incorrect (#623)
TypedNodeWrapper exception with fr:number/currency (#657)
Regression: Don't enable debug XForms logging by default (a83d11197e)
TinyMCE .htm files missing in the build (#663)
java.io.NotSerializableException upon restarting Tomcat (#644)
Upload shows selector if there is an initial file (#674)
xf:upload file size is not formatted (#575)
Unsupported event context information for event('input-only') (#665)
Bookcast example is garbled (#655)
Data type not applied as CSS attribute to xf:input inside a xf:group (#659)
Exception with simple xxf:control="true" (#680)
On IE a space is "removed" from input fields (#682)
xxforms:if / xxforms:while don't work anymore (#684)

Liferay support

Check status of portlet examples (#608)
Some hidden controls appear (#539)
New form crashes in Liferay (#537)
Bookcast fails when saving (#538)
Crash when coming back from accessible mode (#668)
com.liferay.portal.NoSuchRoleException (#305)
Error downloading attachment with proxy portlet (#559)

Other

Restore XUpdate processor (#355)
Option to hide Orbeon Forms version in page (#676)
Tentative fix for deadlock in lastModifiedImpl (f054f8e02d)
cosmetic UI fixes
refactoring

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.beta.201301101955-PE.zip
CE version: orbeon-4.0.0.beta.201301102009-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Today we released the second Beta release of Orbeon Forms 4.0! Like 4.0 Beta and the previous milestone builds this is not a final release.

In this release, we fixed a number of Internet Explorer issues (in particular for IE 7, which we don't recommend to use but which we still support, but also some IE 8 and IE 9 issues), some improvements to the Form Runner Wizard view, and more. Here is the overall list of changes since the first Beta:

Form Builder

Also add Italian and Korean among top languages

Form Runner

Wizard: truncate long section titles to the left (#711)
Non-relevant sections show in wizard view (#699)
Blank oxf.fr.default-logo.uri causes invalid request (#713)
Call the search API without app/form returns no documents (#709)
IE8: licence plate digits appear vertically (#705)
IE7: Navigation boxes around arrows not tall enough (#723)
IE7: Navbar is too high(#721)
IE7: Wizard: too much space in TOC (#722)
IE7: Unbalanced padding with input fields (#727)
IE7: Bootstrap add-on misaligned (#728)
IE7: Grid cell content not top-aligned (#729)
IE7: fr:box-select too wide (#730)
IE7: Upload control too wide (#731)
ClassCastException when using Oracle persistence layer on JBoss 7 (#726)

XForms engine

Regression: XML parsing error with IE9 after fix to #682 (#717)
IE9: Checkboxes misaligned (#704)
xf:output with date type outputs timezone (#689)
Better radio/checkboxes alignment

Other

Set Xerces SecurityManager (#686)

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.beta2.201301160348-PE.zip
CE version: orbeon-4.0.0.beta2.201301160141-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Photo by S@Z

With XForms, you use XPath expressions to specify complex validation constraints and calculations. Take the following XML snippet:

<item>
    <units>3</units>
    <price>50</price>
    <total/>
</item>

You can declaratively calculate the total like this:

<xf:bind ref="total" calculate="../units * ../price"/>

../units and ../price are paths that refer to the XML elements with those names. This is much like a good old spreadsheet formula, except you are not working on cells but on XML data.

Because there is a multiplication with *, each side is atomized by the XPath evaluator. This means that the expression engine looks at the XML elements and extracts a value from them. This makes sense, because you can’t multiply two elements, you can only multiply two numbers!

But there is a twist, because multiplication is defined on different kinds of numbers: in XPath, you can in particular multiply integers, floating-point numbers, and decimals. In this example, what does the XPath evaluator do? Well, by default, it looks at each number as a double-precision floating-point number (double).

And this is not always what you want. For example, when dealing with currencies, you really want decimal numbers in order to avoid funny rounding errors. In this case, you must use casts, which make the expression more complicated:

<xf:bind ref="total"
  calculate="xs:decimal(../units) * xs:decimal(../price)"/>

Now XForms also allows you to assign types to XML data:

<xf:bind ref="units, price, total" type="xs:decimal"/>

This annotates all three elements with the xs:decimal type, with the meaning that the value is required to conform to that type. It’s designed so that if the user, via an input field, enters an incorrect value, or fails to enter a value, that value will be marked as invalid. [1]

But with XForms prior to version 2, XPath expressions were not aware of these annotations. This is understandable, because formally XForms only specified support for XPath 1.0, which had a limited set of data types (string, number, and boolean), and which didn’t specify how annotations on a data model should work.

The good news is that XForms 2 officially supports XPath 2 [2], and this gives the XPath evaluator an opportunity to use type annotations properly. So, with the examples given above, the calculate expression, when looking at the values of the units and price elements, can notice that they have an xs:decimal type, and handle them properly as decimal values. The expression remains as lightweight as it was without annotations:

<xf:bind ref="total" calculate="../units * ../price"/>

You might wonder what happens if the value is not of the specified type. Say you have:

<item>
    <units>3</units>
    <price>foo</price>
    <total/>
</item>

The answer is that when the XPath evaluator tries to access the typed value of price, the calculation is interrupted and its result is set to an empty string. It makes sense not to attempt to complete the calculation in this case, following the “garbage in, garbage out” philosophy.

Orbeon Forms has experimentally supported exposing type annotations to XPath for years, but recently we have ironed them out and enabled them by default for new forms created with Form Builder. And we are now working to officially make this part of XForms 2!

For more details, see the Orbeon Forms documentation.

Besides validation, types can also, depending on the XForms engine, affect the visual representation of a form control associated with that piece of data. A classical example is a date picker for an xs:date value. ↩
Orbeon Forms on the other hand has always supported XPath 2 as an extension of XForms 1.1. ↩

Today we released the third Beta release of Orbeon Forms 4.0! Like 4.0 Beta 2 and the previous milestone builds this is not a final release.

In this release, we fixed some more IE 7 issues, improved Section Templates, the Wizard, and XML Schema generation, and more. Here is the overall list of changes since Beta 2:

Form Builder

StackOverflowError when inserting section template with repeat (#725)
xxforms:get-request-parameter() doesn't work in Validation Details (#742)
Validation Properties: XPath validation uses wrong namespaces (#743)
Crash with XBL containing a bind with id (#770)

Form Runner

Wizard: fix event handler showing first relevant section (453f257d5a)
IE: Autocomplete menu closes almost immediately (#734)
IE7: Summary page: search field too wide (#735)
On IE, on `xf:setfocus`, client tells the server again the focus has been set (#747)
Exception with form using the dialog editor (#760)
Repeat in section template: controls resources are not included (#757)
Section template with repeat doesn't write data back (#740)
Issue when generating the PDF for a form with hidden sections (#756)
PDF: CSS fixes (93f78aaddd)
Wizard: shows non-valid section when invalid value is non-relevant (#778)
Wizard: CSS fixes (0867a2978e)
Schema generation failing when using the MySQL persistence layer (#772)
Schema generation outputs items from itemsets (b63a25bc19)
xxf:default doesn't run in section template (#786)
The resource persistence layer raises an error when saving XML documents (#737)

XForms engine

Event handler with incorrect `ev:observer` associates with parent (#739)
NPE in xf:load with AVT (#746)
Data type not applied as CSS attribute to xf:input inside a xf:group (#659)
xxf:default result is ignored in XBL mirror instance (#787)
xf:recalculate: support xxf:defaults="true" with xxf:deferred="true" (34471c392d)

Other

Upgrade to Ehcache 2.6.3
Upgrade Saxon with backported Numberer_it (fixes Unicode for Italian numbers, dates and times)

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.beta3.201301300219-PE.zip
CE version: orbeon-4.0.0.beta3.201301300332-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

We just added a new sample form as part of our process of testing Orbeon Forms 4.0 beta builds. This is the W-9 form, which is a common US form to request a taxpayer identification number.

In the browser

It shows a number of features, including:

the (new in 4.0) wizard view
how to show and hide controls depending on conditions
how to fill-out a PDF from a template

PDF version

You have a lot of freedom when bringing a form to the web from a paper (PDF) version: adapt the fields, general organization of the form and workflows to help the user make choices quickly!

Today we released Orbeon Forms 4.0 Release Candidate 1 (RC1)! Like 4.0 Beta 3 and the previous milestone builds this is not a final release, but we are really getting there!

Here is the overall list of changes since Beta 3:

Form Builder

Edit Source: don't allow empty document (#815)
IE9: Buttons editors in Builder cause crash (#812)

Form Runner

Currency/number prefixes don't show in view mode (#783)
PDF template: xf:output not supported (#792)
Include W-9 sample form (#791)
Fix possible XPath error when saving attachments (553c0d145c)
Samples: remove unneeded buttons (686b945264)
fr:grid doesn't evaluate remove-constraint in context of iteration (#795)
Controls form: i18n issue with section template (#738)
Controls form: Dynamic Data Dropdown needs some data (#650)
IE7: Navbar content misaligned in Noscript mode (#733)
Upgrade to TinyMCE 3.5.8 (09b185a35c)
Toolbar images of the Formatted Text are not shown, under proxy portlet, on IE9 (#798)
View mode: incorrect breaking of text (#804)
Bootstrap and Orbeon styles have effect outside the portlet (#805)
MySQL: free text search search doesn't return any results (#821)
Captcha doesn't show (#822)
Recaptcha doesn't work anymore (#825)
IE9: fr:us-phone shows "undefined" in field (#810)
Lots of small CSS fixes

XForms engine

Update of switch label with for updates wrong label (#793)
XPath error when hiding section (#799)
Chrome: JS error when going back to Form Runner home (#716)
Upload gets stuck on IE10 (#802)
Occasional NPE on portlet initialization (#819)
Can't focus on captcha (#823)
Setfocus on XBL with mode="focus" focuses on nested control (#824)
IE7/IE8: JavaScript placeholder with initial focus doesn't remove placeholder (#811)

Other

Coding error for socket timeout (#801)

More information is available in the in-progress release notes for 4.0.

You can download the builds using these links:

PE version: orbeon-4.0.0.rc1.201302120214-PE.zip
CE version: orbeon-4.0.0.rc1.201302120301-CE.zip

Don't forget to grab a trial license for the PE version.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

Photo by Jason Pearce

Consider the following XML document in an XForms instance:

<xf:instance id="my-book">
    <book id="simple_book">
        <chapter id="chapter_1">
            <title>Chapter 1</title>
            <para>Hello world!</para>
        </chapter>
        <chapter id="chapter_2">
            ...
        </chapter>
    </book>
</xf:instance>

Some key elements in this document have id attributes which identify the element they are placed on. This means that if you have the value of an id, you can find the associated element. For example the following XPath expression finds the second chapter element:

//*[@id = 'chapter_2']

The downside of using plain XPath here is that if you have a very large document, the expression might have to scan large portions of the document, possily the entire document. This can be costly. In theory, the XPath engine can back the XML document with indexes, but this is typically done only by XML databases such as eXist.

However XPath specifies an id() function exactly for the purpose of improving on searching element with a given id. It is in fact very similar to the JavaScript getElementById() function now implemented natively (and efficiently) by web browsers.

The id() function is not strictly required to be faster than a plain document scan, but it can obviously be backed by an index (a hash map), and this is exactly how we just implemented it![1]

The index is enabled only for a specific instance with:

<xf:instance id="my-book" xxf:index="id">

When support is enabled for an instance, all ids are indexed the first time the id() function is called. Then, for each mutation of the document (insertion, deletion, replacement, change of value), the index is updated. [2] At this time, only elements with a local name of id are indexed. This covers plain id and xml:id attributes.[3]

You can use the id() function this way:

<xf:insert
  ref="id('chapter_2', instance('my-book'))"
  origin="xf:element('chapter', xf:attribute('id', 'chapter_3'))"/>

This XForms action inserts a new empty chapter element with id chapter_3 after the chapter element with id chapter_2.

Form Builder benefits from this change a lot, because:

The entire form being edited is stored in an XForms instance.
All controls and other important elements are assigned ids.
Finding elements by id is a very common operation.

This is available in nightly builds and will be part of Orbeon Forms 4.0.1.

Until this change, calling id() on regular (not readonly) instances in Orbeon Forms always returned an empty result. However, calling it on readonly instances already behaved as expected, as we are relying on the Saxon TinyTree implementation for readonly instances. ↩
The implementation is just slightly non-trivial, because the spec says that when more than one element has a given id, the first in document order must be returned. So we must properly track multiple ids for that reason. ↩
In theory, id() must support any attribute or element with type xs:ID but Orbeon Forms doesn’t implement this yet. In the future, we might also support indexing elements by other aspects, such as CSS classes or element names, as HTML does. ↩

After more than 2000 commits, 19 milestone builds, 3 betas, and one release candidate, we are proud to announce the final release of Orbeon Forms 4.0!

As a goal for 4.0 we wanted to create a release we care about, on the outside as well as the inside. So in this release, in addition to new features and bug-fixes, we have fixed countless UI issues, refactored a lot of code, and thought a lot about architecture.

So what's new since the last major version, Orbeon Forms 3.9? The answer is: a whole lot!

1. A brand new look

Let's start with one of the most obvious changes. Form Runner and Form Builder feature a new look based on Twitter Bootstrap. Forms are more pleasant out of the box, and chances are that more developers will be familiar with Bootstrap when customization is desired.

New look and feel with Bootstrap

2. Security and access control

Orbeon Forms 4.0 introduces many security improvements:

All services are protected by default.
The built-in eXist database is protected by default.
All uses of cryptographic algorithms have been reviewed and improved (blog post).
File uploads are now tamper-resistant (blog post).
Ajax requests are better protected against DOS attacks.
Processing and loading of external DTDs when XML validation is turned off (which is in most cases) is disabled by default.
The product version number is hidden by default.
You can protect your Form Runner forms with a captcha (reCAPTCHA or SimpleCaptcha) (documentation, blog post).
Form Builder supports setting fine-grained CRUD permissions for individual forms (blog post).

3. Repeats in Form Builder

This is one of the most important features of the release: Form Builder now supports grids with repeated rows. See this blog post which introduces the feature.

Example of repeated content

4. Nested sections and live controls in Form Builder

Under the hood, Form Builder had major architecture changes. In 4.0, everything you see in the builder is dynamic. This allowed us to add support for nested sections, and also to let controls (including your own custom controls built with XBL) appear in the builder as they will when the form is deployed. This is also a strong foundation for future improvements to the builder.

Examples of XBL controls at design time

5. Multi-page "wizard" view

The wizard view is a nice new way to present a form to the user, as a sequence of multiple screens. In fact, chances are you will want this mode for most forms! See this blog post which introduces the feature.

6. Improved portal support

Both the full portlet and the Form Runner proxy portlet have been improved and tested with Liferay 6.1 GA2. Other improvements include:

Liferay role and user information (including email and full name) is available to the full portlet (documentation).
The workflow-review/workflow-send buttons work with the Form Runner proxy portlet.
The Orbeon Forms CSS is sandboxed so that the portal style is not impacted.

Form Runner in Liferay 6.1

7. New web site

We have simplified and modernized our web site for the 4.0 release. Check it out! From there you can find out more about Orbeon Forms 4.0, how to try it out, and how to get support.

8. And much more

There are too many new features to describe in this post. For more details, see the Orbeon Forms 4.0 release notes, which also covers backward compatibility.

We hope you enjoy this release!

Say you created two forms with Form Builder: a contact form and a phone call form. In the phone call form, you use one of the fields to capture the name of the person you'll have a call with. Instead of using a text field, you'd like to have a dropdown, with the values in the dropdown coming from values previously captured with the contact form.

Form Builder doesn't come, out-of-the-box, with a way to link forms in the manner described above. However, this is something that you can do with Form Builder's Services & Actions. The idea goes as follows:

Write a query retrieving the contact names captured with the contact form. The query will differ depending on the persistence layer you're using: it will be in XQuery if you're storing data in eXist, or SQL if you're storing data in Oracle or MySQL.
Create a service that runs this query. It will be either an HTTP service (to run XQuery) or a database service (to run SQL).
Add a dropdown to your form.
Create an action that runs the service on form load, and use the result to populate the dropdown.

The query

For the purpose of this post, we'll assume data is stored in eXist, which is bundled with Orbeon Forms, and we'll query data in the contact form, which is one of the sample forms. Since we're querying eXist, we're using XQuery:

for $c in /form/contact return element contact {
    element label {concat($c/first-name, ' ', $c/last-name)},
    element value {tokenize(util:collection-name($c), '/')[last()]}
}

This query returns one <contact> element per entry in the contact form, with for each one, the person name (to be shown in the dropdown) and an id (stored in form data, so we can link a phone call with a person). The root element of data captured with forms you create with Form Builder is always <form> and in the contact form the first/last name are in a section named contact, so the query iterates over /form/contact. Also note how the query retrieves the document id in which a node $c is stored with tokenize(util:collection-name($c), '/')[last()]. With this, the query returns <contact> elements that look like:

<contact><label>Homer Simpson</label><value>9eff349bfd95aab8d4d5e048bd25a815</value></contact><contact><label>Charles Burns</label><value>138b27e80eefc3f0429a3d14cbc77df527630787</value></contact>
...

The service

You can pick a name for the service, say get-contacts. The URL should point to the contact form, as it is stored in eXist. With the embedded eXist, if Orbeon Forms is deployed on http://localhost:8080/orbeon then the URL pointing to the contact form will be (here the URL is split in two lines for formatting, when pasting it in Form Builder put it on one line with no spaces):

http://localhost:8080/orbeon
  /exist/rest/db/orbeon/fr/orbeon/contact/data

The service POSTs and XML document as follows, with the XQuery above within the <exist:text> element.

<exist:query xmlns:exist="http://exist.sourceforge.net/NS/exist" max="0"><exist:text> ... </exist:text></exist:query>

The action

Finally, the action calls the service on form load, and uses the result to populate the dropdown, as shown in the screenshot on the right (click it to see a larger version). Click the test button, and you'll see how the dropdown populates with values entered previously in your contact form.

Wrapping it up

Since populating a dropdown with values from another form at this point isn't supported out-of-the-box by Form Builder, to do so you need to write a query against the database and use services and actions to run the query and populate the dropdown. While this isn't as simple as we'd like it to be, and future versions of Orbeon Forms might make this process simpler, it is already something that you can do today, and it goes to show the flexibility of the services and actions feature, which allows you to query existing data, whether it has been captured with a form you created with Form Builder or by some other mean.

Today we released Orbeon Forms 4.0.1, a bug-fix and performance release.

Here is the list of changes since 4.0.0:

Form Builder

Open/close state of existing sections is lost when inserting new sections (#177)
Itemset editor: fixup label/value if needed (#830)
Close large form sections upon load to improve performance (#838)
CodeMirror gets unusably slow with very large forms (#840)
Service body lost after hitting save (#859)
Validation Properties shows, but doesn't contain properties (#861)

Form Runner

fr:section shows closed arrow when becoming relevant (#862)
Empty, non-closable dialog upon PDF error (#864)
Property to allow PDF generation even if form is invalid (#871)

XForms engine

MIP evaluations are slow with many bind/@name (#834)
Failing constraint should default to false() (#835)
Support for XPath id() function (#836)
Support xxforms-replace event (#837)
Order XBL JS and CSS resources in consistent order (#844)
Can't access event('response-body') if debugging is disabled (#863)
Value of xf:textarea mediatype="text/html" not updated on "switch" (#868)
AssertionError upon xforms-submit-error (#876)
Configurable xf:input/xf:textarea string replacement (#855/#867)
Lightweight syntax for HTML LHHA (#874)

Other

Blank page-public-methods property causes default to GET HEAD (#865)

You can download the latest version of Orbeon Forms from the downloads page.

Please send feedback:

via twitter
to the ops-users list
or as comments on this blog entry

We've written before about the importance of having a set of comprehensive unit tests for Orbeon Forms. For instance, we put in place the necessary infrastructure to test code in Scala back in 2011 and to test XML pipelines back in 2005. With this, every time a check-in is made, a build is automatically created and 687 tests (and counting) are executed. If any test fails, we get a notification, and fixing the build becomes our priority.

But until now, we couldn't automatically run tests that requires a browser, which means that a significant portion of Orbeon Forms couldn't be automatically tested, including all our client-side code (JavaScript and CoffeeScript), interactions between client-side and server-side code, and higher level components such as Form Builder and Form Runner.

We now have this infrastructure in place:

Running the tests automatically– Everytime we do a check-in to our Github repository, Travis-CI checks the code out, compiles the code, runs the tests, and sends us an email if any test fails.
Running Tomcat from Ant– We still use¹ the venerable Ant as our build system, so it is up to Ant to compile the code and run the tests. The new tests require a server to run, and for that we use Tomcat. During the build, we install Tomcat in the build directory, and configure it to pick up its configuration files from test/catalina_base, by setting the catalina.base property. Tomcat needs to run in parallel with the tests, which we do with Ant's Parallel task. Before running the tests we need to make sure Tomcat is successfully started, which can be done very easily with Ant's Waitfor task. To see how this is done, check the target name="test" in our build.xml.
Starting a browser– Our tests run in a real browser: Firefox². Since the test machine isn't connected to an actual screen, we run Xvfb through our .travis.yml.
Writing the tests– To drive Firefox, we use Selenium 2 with WebDriver. Our tests are written in Scala, with ScalaTest and its Selenium DSL.

And in case reading that we are still building on Ant makes you worry about our sanity, rest assured that we are thinking of moving to a more modern build system, like sbt.
ChromeDriver currently can't be used on Tavis-CI as Chrome does not work in an OpenVZ container, which is used by Travis-CI. But in the future, while still using Travis-CI, we might run the tests on more browsers through Sauce.

Variables in XForms 2

Orbeon Forms 4.0 M16

Orbeon Forms 4.0 M17

Simplify your queries with Oracle's dense_rank

Form Runner Wizard View

Orbeon Forms 4.0 M18

Creating REST services with Google Apps Script

Orbeon Forms 4.0 M19

Automatic remapping of Windows-1252 characters to Unicode

Orbeon Forms 4.0 Beta

Orbeon Forms 4.0 Beta 2

Better formulas with XPath type annotations

Orbeon Forms 4.0 Beta 3

New sample form: W-9

Orbeon Forms 4.0 RC1

Faster XML search with the id() function

Announcing Orbeon Forms 4.0

Dropdown populated based on values from another form

The query

The service

The action

Wrapping it up

Orbeon Forms 4.0.1

Automated browser testing with Selenium on Travis-CI