Summary

This chapter details the use of the Perl scripting language within Arbortext APP.

1 

Introduction

Combining Perl with Arbortext APP's existing scripting capabilities means that the possibilities for automated solutions are hugely increased.  This section describes Perl scripting features in Arbortext APP and provides practical examples of Perl scripts.

This document assumes that the reader has some prior knowledge of Perl and Arbortext APP scripting mechanisms therefore the purpose is to provide a specific description of Perl in Arbortext APP.  This is not a tutorial for programming with Perl. It is assumed that the reader is familiar with Arbortext APP and it is recommended that you read the Arbortext APP Scripting section before reading the Perl Scripting chapter.

2 

Installation

To install Perl simply follow the install instructions from the installation menu on the Arbortext APP CD as described below:

or

3 

Integration

Now that the advantages of Perl have been incorporated, you can:

There is also full access to the complete library of Perl modules and the additional modules made available.

3.1 

Reentrancy

Pre version 8 it was possible for Perl code to call back to Arbortext APP with macro() and for that code to call back to Perl, and so on without limit. This was found to cause problems with stream data being corrupted. The fix is to throw a Perl error if you attempt to call back to Arbortext APP from within a Perl eval or channel. The error message you would see is:

But you are currently able to call back to Arbortext APP from within a mini script.

4 

Implementation

The original Arbortext APP Perl implementation (described in Technical Note 0038) ran each Perl script in a separate thread, and communicated with Arbortext APP using variables. This implementation caused problems on multi-processor machines and so was changed.

The version 8+ implementation of Perl now runs a Perl script in the main Arbortext APP thread and directly runs Arbortext APP macros from the Perl interpreter.

The changes are minimal and existing Arbortext APP Perl scripts will work in the same way as before, however there are some changes to be aware of. The first change is that Arbortext APP can only respond to the user when the Perl script calls back to run a macro or read from a stream.

If you write a Perl script that spends a long time processing some data without calling back to Arbortext APP, then the interface will not respond to the user. To allow user interaction while a script is running, a subroutine has been added that will allow Arbortext APP to check its event queues and process any macros issued by the user:

As Perl scripts are no longer run in separate threads, you can only have one script running at any one time. A Arbortext APP script can still call a Perl script, and visa versa, but only the top level script will ever be running at one time.

4.1 

Perl versions and modules

3B2 versions come with their own version of Perl supplied that should be installed prior to the Arbortext APP application, however you can use an alternative version of Perl if need to, such as ActiveState for example.

In your sargsw.3ad Arbortext APP system file, there are a number of arguments for 3B2. The most common one will be '-p:Printer' to denote what printer you use. To tell Arbortext APP to use a different version of perl, use '-R' and point to your perl dll. For example:

One reason you might want to do this is to use a specific perl module of a Perl version, for example acessing databases could be done this way, and you could use Win32::ODBC to access your database.

4.2 

Perl Snippets (V9)

Perl snippets are a new way to speed up Perl prossesing in documents which make heavy use of snippets. It works by turning the 'perl snippet' into an anonymous subroutine, and caching it. Arbortext APP will then store a mapping between the snippet and the subroutine. Each time a snippet is to be run we check to see if we have already compiled it into a subroutine, if not we create it, but either way it will be run. The result of calling the subroutine is the same as just evaluating the perl snippet.

They are controled from the 'speed' tab of the document preferences diologue box tdpref, or by using the perl_mode keyword.

Snippets apply to the 'perl' function and the 'perl' macro e.g.:

It DOES NOT apply to perl streams e.g.:

Snippets are intended to be transparent to the user; that is, whether the feature is on/off should not affect the results from calling into perl from APP. The feature is enabled on a per document basis. If no document is open, and a call into perl is made then the 'compiling' is not done. The cache is a global one and is cleared when any document is closed to prevent it growing without limit.

5 

Types of Perl Script in 3B2

5.1 

Document specific Scripts

Perl scripts within Arbortext APP documents are simply text streams, and can be entered in the edit bar or stream editor in the same way as Arbortext APP scripts. An internal Perl script is a script set in a normal text stream within the document and appended with the .pl tag extension.  It is recommended that you always use this extension for Perl scripts because Arbortext APP will then know to use the Perl interpreter when the script is executed.

When entering or editing a script, remember that you must stick to the rules of Perl syntax, for example, do not forget to include semi-colons at the end of statements. You may prefer to to develop or edit your perl scripts in a recognised perl syntax supporting editor and tester. To create a perl script stream enter the following in the macro bar:

The extension .pl tells Arbortext APP to use the Perl interpreter when running this script.

By creating a script within your document, you will easily be able to develop and test the script. It also means that the file is portable because regardless of where the file is located, the script will always be inside your document. The drawback with this method is that only the file containing the script will be able to use it.

5.2 

External String Scripts

External String Perl scripts are simply Perl scripts set into an external strings file, using a defined set of user string numbers. These files are generally located in the Arbortext APP system directory and read into Arbortext APP on start-up.  Scripts located in strings can be used form any document, or even when no document is open.

6 

Running/Executing Perl Scripts

6.1 

The trun macro

The trun macro controls the running of scripts. It can be used to start and stop both types of script (stream and string), either manually by the user by being entered in the macro bar, or automatically from within a script, for instance, the following syntax would run the Perl script called "my_script":

The above example runs as a Perl script if the script has the file extension .pl or as a Arbortext APP script otherwise. It is recommended that you always use the .pl extension for Perl scripts because Arbortext APP will then know to use the Perl interpreter when the script is executed. However, if the script does not have the .pl extension then you will have to indicate that it is a Perl script by placing a 'p' in front of the script name.  It is therefore possible to use the following syntax to always run a script as a Perl script:

6.1.1 

Passing Parameters

Perl scripts can be passed a single string parameter in the same way as Arbortext APP scripts. This will appear in the standard Perl @_ array as element 0 (access it from Perl with $_[0]). There is no limit to the size of this parameter.

Perl scripts are usually run in one of two ways described in the following sections.

6.2 

Running internal perl scripts

Use the trun command followed by the name of your script in double quotes, for example:

6.3 

Running external perl scripts

To execute an external string file use the trun command followed by the string number of your script.  Prefix the string number of your script with a 'p' to indicate it is a Perl script, such as:

6.4 

Stopping a Perl Script

To stop a Perl script, use the standard trun -1 or trun 0 macros.  However, this will only be effective the next time the script makes a service request from Arbortext APP. If you write a script which never makes such a request then you will not be able to stop it from Arbortext APP.

7 

Developing a Script

7.1 

Perl Script Components

7.1.1 

Variables

7.1.1.1 

Shared variables

Arbortext APP and Perl share the same variables. The implementation of variables in Perl is more comprehensive than that of Arbortext APP; therefore the Perl interpreter is responsible for all named variables.

That is, 3B2's variable ^varname is the same as Perl's $varname. Changes to one are reflected in the other.

7.1.1.2 

Case Sensitivity

All variable names are case sensitive. This means that previous Arbortext APP scripts that refer to ^variables with inconsistent case will not work.  Use the following search and replace to find all references to ^variables in any text stream and optionally replace them with lowercase names:

This will not catch any variables without the ^ prefix, such as with the getvar macro: getvar 00000 'variable'. This means that you will have to write another search and replace for that. Alternatively, write a stand-alone Perl script to scan entire Arbortext APP documents for inconsistent variable usage.

7.1.1.3 

Arrays

7.1.1.4 

Lexical and Global Variables

All variables defined or called in a script are global variables which are stored as Perl package variables. Global variables exist during the whole Arbortext APP session.

Lexical variables are only available during the runtime of a Perl script.  They cannot be accessed from a Arbortext APP script.  It is possible to define variables and arrays as lexical using the Perl 'my' keyword.

You can define a global variable with the same name as a lexical variable.  The global variable will not be influenced by manipulating or defining a lexical variable with the same name.

7.1.1.5 

Accessing Perl Variables

Normal Variables

Perl splits its variables into three different types: package variables, lexical variables and special variables.  Lexical variables are declared using 'my';. Special variables are those like @ARGV and %SIG (see Special variables generated through startup of Perl) which mean something special to Perl.  Otherwise it is a package variable, for example:

To access normal Perl variables or array elements (package variables) from Arbortext APP, simply use the existing syntax. The following are examples of using Perl variables with Arbortext APP syntax and next to each example is an explanation of the syntax:

Explicitly Named Package Variables

To access unusually named Perl variables such as those with fully qualified package names or the special punctuation variables, use the quoted name syntax, for example:

8 

Perl Communication with 3B2

8.1 

Calling Arbortext APP functions from Perl scripts

8.1.1 

The macro Perl function

It is already possible to call macros from Arbortext APP scripts.  Any macro can also be called from a Perl script by using the 'macro' function, for example to call the refresh macro in a Perl script use the following syntax:

The macro function takes a single string which is passed to the Arbortext APP processor for execution.  Therefore, when calling a Arbortext APP macro that refers to information in double quote delimiters, (like tf "times" for example) it is convenient to use the macro function with Perl's 'q' or 'qq' operators.  Use 'qq' to interpolate embedded variables or 'q' otherwise.

The example below calls the Arbortext APP insert (@) macro and inserts the text 'Insert this Text' at the current cursor position.

This example uses the Arbortext APP ttagmk command to create a new text stream called 'new'.

If the function involves using a variable, use the 'qq' function to interpolate embedded variables.  For instance this syntax makes a tag with the value of the variable called tagname:

8.1.2 

The AUTOLOAD function

Perl allows you to define a subroutine called AUTOLOAD which is called whenever it encounters a subroutine call for which it has not already seen a definition for. (Consult the Perl documentation for more details). Arbortext APP defines an AUTOLOAD subroutine which calls the macro of the same name as the 'missing' subroutine along with its arguments. Start by declaring which macros you will use during the script, using the syntax shown here:

When you have defined the macros you will use in a script with the AUTOLOAD function it is possible to call syntax directly without the need for the 'q' and 'qq' operators, for example:

1

Declares the commands and keywords to use in the Perl script.

2

Re-formats the document using the tformat command.

3

Makes a new text stream called 'new' using the ttagmk command.

4

Creates a variable called name.

5

Runs a script called 'conversion' using the trun command.

8.2 

Calling Perl scripts from Arbortext APP scripts

8.2.1 

The perl command

The perl macro enables Perl to be called directly, outside of any script context. The macro passes everything from the first non-space to the end of the macro line to the embedded Perl interpreter for evaluation.

Below are several examples of using the perl macro:

8.3 

Stream Access from Perl code

8.3.1 

Read

One method of gaining access to a stream would be to use a Arbortext APP variable assignment to handle this with the macro function:

8.3.2 

Write

It is also possible to write to a stream using the variable assignment method used in Arbortext APP, for example:

8.3.3 

The tie function

Another way to access Arbortext APP text streams is provided via Perl's tied file handles. The Perl function tie binds a variable to a package which then implements various special subroutines.  In order to write the content of a text stream or reference in a Perl variable or to write the content of a Perl variable in a Arbortext APP element, a suitable implementation has been provided via the 'Perl3B2::Handle' package.

For example, to write the content of '$var' to the text stream 'txt1' use the following syntax.  It is described afterwards line by line:

6

Define a file handle 'TXT' using the 'Perl3B2::Handle' package.  (Replace 'TXT' with the name of your choice).

8

Open 'TXT' and associate it with a text stream 'txt1'.  It will be created if it does not exist.  In this case > stands for overwrite and >> stands for append.  If 'TXT' can not be opened the script will stop (die) with an error message.

10

The next step is to write the contents of $var to TXT.

12

Close the TXT file handle that you have opened.

The method above writes the content of txt1 line by line.  If the content of txt1 has multiple lines it is possible to read or write the content line by line using a Perl while loop.  For example, this next example reads the content line by line:

1

Define a file handle 'ABC' using the 'Perl3B2::Handle' package.  (Replace 'ABC' with the name of your choice).

2

Open the file handle 'ABC' and read the contents of txt1.

3-5

If the content of txt1 has multiple lines you can read it line by line using a while loop and adding each line to $var.

6

Close the 'ABC' file handle.

Alternatively, it is also possible to read the content in one single step using the following method:

1

Define a file handle 'IN' using the 'Perl3B2::Handle' package.  Open the Arbortext APP stream called 'stream' and tie to the filehandle IN.

2

The Perl seek function enables you to specify where the script reads the contents of the stream from.  'seek IN, 0,2;' moves the file pointer to the end of the file.

3

The Perl 'tell' function returns the current position of the last read file handle.

4

'seek IN,0,0;' moves IN's current position back to the start of the file.

5

Read $len number of characters from IN (starting at current position) and copy the contents into a variable.

6

Close the 'IN' file handle.

The example below demonstrates writing the result of a diamond operator in a stream:

8.3.4 

Resolving APP variables from Perl (V9)

The only existing method for properly resolving Arbortext APP variables inside Perl functions pre version 8.13h, involved resolving the variables within Arbortext APP and passing them to Perl using the shared script variables.

The new "var3b2" Perl function, available automatically via Perl3B2, allows a Perl script or miniscript to retrieve or modify any valid Arbortext APP variable type including such items as function variables, strings and FISH.

Be careful to avoid resolving ^[perl ..] function variables from var3b2, this works in theory but may become unpredictable for complex cases. For example:

Example

8.4 

File streams

8.4.1 

STDIN STDOUT and STDERR

Three file streams have special properties in Perl:

These file streams are described in the following sections.

8.4.2 

STDIN

STDIN (standard input) is the default source of input.  This is the file associated with a keyboard.  STDIN is where typed data appears when a Perl program is running.

8.4.3 

STDOUT

STDOUT (standard output) is the default destination for output.  In a stand-alone Perl script, sending data to the STDOUT or STDERR, file handles will cause it to appear in the console window.

As there are no console windows within Arbortext APP's embedded Perl, STDOUT and STDERR are redirected to the tied file handle package 'Perl3B2::StdHandle' by default.  The output goes to the actual cursor position in Arbortext APP by invoking Arbortext APP's @ @ macro to enter the text.

It is also possible to redirect STDOUT to any Arbortext APP stream for example:

8.4.4 

STDERR

STDERR is used by convention as the error stream of nearly all external modules.  STDERR (standard error) is the default place to send warnings and errors.

The default of this stream also goes to the actual cursor position. To redirect it, use the same syntax as shown for STDOUT above.

To reset STDERR to default use:

9 

Debugging a Script

Perl has excellent error reporting and comprehensive debugging facilities.

For more information on the Stream Editor see Dialogue Boxes and ttagedit

9.1 

The Perl Script Information Toolbars

9.1.1 

The Perl Bar

Before you write any complex Perl scripts it is a good idea to enable the error reporting from Perl. The Perl script information toolbars display Perl script information including error reports.

To enable the Perl bar:

Alternatively, enable the perlbar using the toolbars macro as shown below:

There are two boxes on this toolbar:

9.1.2 

The Script Bar

There is an additional Perl script information toolbar that displays more information about nested and running scripts.

To enable the Script bar:

Alternatively, enable the script_bar using the toolbars macro as shown below:

There is more information displayed on this toolbar:

You can also disable APP debugging functions to speed up formatting using the perl_mode.

9.2 

Comments

It is recommended that you add comments to your scripts. Comments enable you to know what each piece of a script is doing.  Perl will ignore comments in a script and just process the Perl statements.

In Perl, comments are as shown below:

A # comment instructs Perl to ignore everything from the # up until the end of the line.

It is also possible to have a comment on the same line as other Perl statements, for example:

9.2.1 

Multiple Comment Lines

It is is also posible to have multiple lines of comments. This provides embedded Perl documentation. Multiple lines of comments end with =cut, for example:

10 

Nested Multiple and Chaining Scripts

Just as with normal Arbortext APP scripts it is also possible to run more than one Perl script.  As with Arbortext APP scripts, Perl scripts can be nested. This means that a Perl script can use the trun macro to start another Perl script or Arbortext APP script.

As Perl scripts can have their own named subroutines, you may find that only one Perl script per task is necessary. In other words, there will often be no need to call nested Perl scripts for any given task.

For more information about Nested Scripts see the Scripting Chapter

11 

Perl miniscripts

Miniscripts are scripts you can refer to within an expression to enable string manipulation. They are run directly in paragraph styles or in a text stream.

By running a Perl miniscript a global variable with the same name as the miniscript is generated.  This contains the content of the return value of the miniscript.  The return value holds the last assigned value in the script if explicitly set. 

There are some important points to note about miniscripts, including several restrictions:

11.1 

Running a Perl miniscript

You can call Perl miniscripts in the same way as normal Arbortext APP miniscripts.  Miniscripts are scripts you can refer to within tag paragraph styles or in a text stream. Miniscripts are not executed using the trun command, but are referenced from within your document tags with the following showstring syntax:

To execute a miniscript use the following function variable syntax:

or

12 

Perl function variable

In version 8 a new set of features called function variables was implemented, one of which was 'perl'. This enables execution of single lines of perl script within a showstring, making simple perl functions normaly run via mini script even easier. The syntax for function variables is show below:

13 

Exporting and converting Perl scripts

13.1 

Exporting internal scripts to external scripts

To export an internal Perl script, simply use the tsavetxt macro, for example:

Or just cut and paste the text of your script into your own text file.

13.2 

Converting internal scripts to an external string

You can save internal Perl scripts as external string number scripts with the tsave macro, for example:

It is then possible to modify the fdt-header in an external text editor such as Notepad.

14 

Perl Files supplied with 3B2

14.1 

The perl3b2.3ad file

When Arbortext APP starts, it allocates a single Perl interpreter and uses the interpreter for all Perl operations. During this initial process, Arbortext APP's Perl start-up file, perl3b2.3ad, is executed.

It is this file that provides the 'Perl3B2::Handle' and 'Perl3B2::StdHandle' packages and communication to Arbortext APP via the designated variables in the 'Perl3B2' package. Other Perl initialisation is also performed in this file.

The perl3b2.3ad file provides you with the following functions. If not explicitly mentioned all functions are exported into the namespace main:

14.2 

The perluser.3ad file

15 

Testing and developing Perl outside 3B2

This only makes sense if main parts of your program can be separated in a way they run independently of Arbortext APP.

15.1 

Advantages of external development

15.2 

Disadvantages of external development

16 

Testing and developing libraries or modules

You must develop and test a Perl library or module that communicates with Arbortext APP in Arbortext APP.  However, modules and libraries are linked only once during the runtime of Perl.  Therefore they are linked only once in your Arbortext APP session. This is very good for the performance, but it is annoying to restart Arbortext APP after each change in the module during the development.  One way of reloading a module each time is to use 'do' instead of 'require' and 'use'.

17 

Perl and Unicode

From version 5.6 Perl has full UTF-8 support.  Because Perl uses UTF-8 encoding and Arbortext APP uses UTF-16 encoding it is necessary for Arbortext APP to internally convert characters to UTF-8 to be manipulated in Perl.  Once the manipulation has been performed, characters are converted back to UTF-16.

This process means that Perl scripts take fractionaly longer to complete than in the Arbortext APP standard version.

For further references on Perl and Unicode see the Further Information and References at the end of this document.

18 

Script Examples

18.1 

Script Example 1

This simple script defines two variables and inserts the contents of the variables at the current cursor position.  After the script there is a line by line description:

1

Defines the variable firstname as 'first'.

2

Defines the variable lastname as 'last'.

3

Calls the Arbortext APP @ (insert macro) macro function.   The @ (insert macro) inserts the contents of the firstname variable at the current cursor position.

4

Emulates the keyboard entry of a carriage return (#13) using the # # macro.

5

Calls the Arbortext APP insert macro using the macro function.   The @ (insert macro) inserts the contents of the lastname variable at the current cursor position.

As an alternative to using the macro 'q' function to call the Arbortext APP commands it is also possible to use Perl syntax to achieve the same result, for example, instead of using:

it is possible to use:

1

Inserts the contents of the variable firstname at the current cursor position. The \n is converted to a carriage return.

2

Inserts the contents of the variable firstname at the current cursor position.

18.2 

Script Example 2 - directory listing append and output

This example opens a directory in the same way as opening a file using the 'Perl3B2::Handle' package.  The contents of the directory are then output to the text stream strm0 with a body tag appended to each line.

1

Open the directory c:\Advent\3B2\.  DIR specifies the name of the directory handle.  Check the return value of the opendir to make sure it succeeded. If it didn't the script will die with the message 'can't opendir /Advent/3B2'.

2

readdir DIR returns a list of all the contents of the directory handle DIR.  This also contains the current directory (called '.') and the parent directory (called '..').  Perl's grep function takes an expression and uses this to 'filter' out a list.  The grep function will return the filenames that do not match the pattern specified, i.e. it will return the list of files (except for '.' and '..'). Then, @files is used to contain this list (see number 6).

3

Close the DIR directory handle.

4

Associate/tie the OUT file handle to Perl::Handle

5

Attach the OUT filehandle to Arbortext APP's 'strm0' for writing

7

Send data to 'strm0' via the OUT file handle.

8

The end of the 'while' loop.

9

Close the 'OUT' file handle. The tie is now severed and the Arbortext APP stream exists independently.

18.3 

Script Example 3 - Get Plain Text and Mark-up in XML

The following script shows how to open a text stream using a file handle, then the contents of the text stream are changed and the alterations output to a text stream called 'output'.  After the script there is a line by line description of each component of the script.

In the following example the text stream strm0 contains a list of names defined as such:

The aim of the script is to mark-up the list of names above to the result of the one name detailed below for all.  You will notice that the names have been separated and are now surrounded by 'tags':

To automate this conversion process use the script detailed below:

19 

Further Information and References

19.1 

Internet Resources

For further general and in-depth information about Perl, sources, mailing lists etc:

www.Perl.com

Comprehensive Perl Archive Network - collection of Perl software and documentation:

www.cpan.org

Perldoc.com - Searchable repository of official Perl documentation. Including CPAN modules.

www.perldoc.com

19.2 

Books/Literature

Programming Perl

Larry Wall

Published by O'Reilly

Learning Perl

Randal L. Schwartz, Tom Phoenix

Published by O'Reilly

Perl Cookbook

Tom Christiansen, Nathan Torkington

Published by O'Reilly

Object Oriented Perl

Damian Conway

Published by Manning Publications

20 

Subject Index