Scripting opens up endless possibilities for working with ProRefactor and the syntax tree.
There are various scripting languages available for running on the Java Virtual Machine (JVM). Those include Groovy, Jython, JRuby, Rhino Javascript, and others. Any of those should allow you to access ProRefactor as a library, and run the parsing and analysis programs it contains. I chose Groovy because I found that Groovy had the cleanest interfaces between its scripts and existing Java applications and libraries like ProRefactor.
See Standalone Jar File for setting up ProRefactor for use from a scripting environment.
This script allows you to generate XML for each of your compile units. The XML contains the entire syntax tree, as well as the symbol tables. This XML is a very raw dump of Java objects from memory.
What is the benefit of this? It allows you to use OpenEdge scripts to gain access to all of the useful information that is generated by ProRefactor. For example, you might use the XML as input for ABL scripts to generate code documentation, to augment your XREF indexes, or to allow you to perform ad-hoc search and analysis.
The XML is unlikely to be self explanatory, so please feel free to post questions in the CodeParse Forum.
Ideally, this will inspire a new open-source project, written in ABL, which will read the XML and convert it to TEMP-TABLEs or some other ABL data structure. That would make the data much easier for other ABL programmers to understand it and use.
This script may require a lot of CPU, memory, and disc space. If your application is large, then the XML that gets written may require a very large amount of disk space, even though it is compressed. (For example, it might take as much as a few GB for an enormous application.)
The script writes to the prorefactor/xml directory. The script is easy to change if you want it to write its output elsewhere.
See the comments at the top of the script for usage notes.
Proparse is a DLL (.so on unix), with an API for using it from ABL(4GL) programs.
The XML generated here contains all of the data that is available in the syntax tree generated by Proparse. In addition, it contains all of the symbol tables generated in ProRefactor. Those symbol tables represent a wealth of information about your source code, far beyond what is available from Proparse alone.
This XML output could (should?) be considered a replacement for the old Proparse API. For the most part, new ABL programs should use this, rather than use Proparse directly.
FileRefs.groovy
May 2008 by John Green.
Find and fix code developed on Windows so it has file references which are portable to unix.
Does the following for RUN *.p, RUN *.w, and for include file references:
Copyright (c) 2008 Joanju Software (www.joanju.com). All rights reserved.
Released under the terms of the Eclipse Public License v1.0:
http://www.eclipse.org/legal/epl-v10.html
This Groovy script finds variables which use the same name as a database table or field name. It writes its findings, as it runs, to the console. It also accumulates a sorted report which it writes at the end of its run.
I wrote this as a test case for Groovy scripting for ProRefactor. A better place for it would be as a rule in Prolint.
I decided to write this after seeing a post on the PEG from Hendrik Demol looking for a way to do this. Assuming he doesn't object, Hendrik gets the Groovy script named after him. :)
The file download can be found at the bottom of this page.
See this page, and follow the instructions there, for getting and using the ProRefactor standalone jar file:
Standalone Jar File
See this page, and follow the instructions there, to generate the project settings files necessary for using ProRefactor:
Project Config. Dump
When launching the script, the prorefactor directory, with its project settings files, must be found in the current working directory.
Note that ProRefactor generates large volumes of data files in its prorefactor directory.
The script expects the source code to be found in the current working directory. That can easily be tweaked if necessary, find this:
// Look from the current (working) directory down.
new File('.').eachFileRecurse{ file ->
and change '.' to '/your/directory/' if you don't want to run the script from the top of your source tree.
The basic command line to launch a Groovy script like this via ProRefactor's standalone jar is:
java -Xss2M -Xmx128M -cp prorefactor.jar groovy.ui.GroovyMain hendrik.groovy
You might want to redirect the output to a file, or tweak the output destination of the sorted report output at the end of the run() method in the script.
If you want to tweak this script for your own use, you might want to see the ProRefactor API documentation. See the javadoc page for how to find ProRefactor javadoc online.
Much of the script is scaffolding, for example, finding and loading the ProRefactor project settings files, iterating through compile unit source files and parsing them, etc. There are methods for building sorted lists of table and field names, and looking up names to see if they are unique in those lists.
The run() method is what gets run when the Groovy script is launched. It iterates through all of the .p, .w, and .cls files in the directory.
The checkSymbols() method does the really interesting work of iterating through all of the symbols (variables, frames, streams, etc) in the compile unit, and checking their names to see if they are also valid table or field names.
Utilities for working with the syntax tree from ProRefactor. (Download below)
Prints the tree under the given node, output to stdout.
NOTE: At time of writing, this is simply dumping the token names from Proparse's API. This script should be changed to be sure to get the valid TokenTypes.TOKEN_NAME. See the notes at the top of http://www.joanju.com/dist/index.html.
Test if part of the AST matches a given specification.
The specification is a list. Each element in the list may be an integer node type, a closure, or another nested
list describing the previous node's children.
Example:
import org.prorefactor.core.TokenTypes as T
[
T.RUN, [
T.VALUE, [
T.LEFTPAREN,
T.Widget_ref, [
T.THISPROCEDURE,
T.OBJCOLON,
{it.getText().equalsIgnoreCase('PRIVATE-DATA')} ],
T.RIGHTPAREN ],
T.PERIOD ]
]
Note how the example looks a lot like a simple printout from printTree():
RUN
VALUE
LEFTPAREN
Widget_ref
THISPROCEDURE
OBJCOLON
ID "private-data"
RIGHTPAREN
PERIOD
Also note that in the example, we are using a Groovy closure to test a node's text, rather than the node type. Anything from the node could be tested, like filename, line, etc. (The "it" variable is the implicit single argument in a Groovy closure, see the Groovy User's Guide for details.)
Once "import static" is supported in Groovy, we won't have to write "T." for every node type, and the specifications will be easier to read.
A null in the list will match any node. An empty sub-list will cause a node's children to match without checking.
The first element of any list must not be a sub-list.
Typically, the input spec list will contain two elements: a spec for a head node, and a sub list containing the specs for its children. But, it's also perfectly valid to specify n sibling nodes in the spec list, and maybe their children.