Parsing test
James and I are continuing to have NullPointerException problems. I wrote a program which just goes through the files in a directory tree and does a Proparse parse of them, with the hope that this would show us the files with parsing errors. *But*, we are getting no errors from that. But, in the first ABL2DB task where we use Proparse, we are getting the NullPointerExceptions. The only thing that occurs to me as an explanation is that Proparse successfully builds the parse tree, but there is something wrong with it so that when we start to navigate the tree it steps in a hole. Does that seem possible to you?
If so, what would you recommend as test to extend the existing scan which would help find the holes?
Now that I am finally back on this, we only have about a month before PCEMEA and I would really like to have James code as a part of the talk.
- Login to post comments
Re: The decimal procedure names
Looks like I better add support for it.
The problem is with what the AppBuilder is doing. If the AppBuilder put the quotes around the whole thing, like "EXCLUDE-S&SPreparse", then it probably would not be a problem. But as it is, Proparse sees this as multiple tokens within the DEFINED() function. Adding support for this in Proparse is doable but a bit tricky.
I had been suggesting that
I had been suggesting that the right thing to do was to eliminate it, but given the number of programs and the number of instances within each program, that is non-trivial. And, of course, being AppBuilder code, one can't just fiddle with it without breaking something.
In terms of frequency I
In terms of frequency I think they're all one-offs. So not hard to remove. Except for the example I've posted up this morning which doesn't return from parsing. That one I've got about 5 examples of and I'm not all the way through the code base yet.
procedure name
I have to ask... why is the procedure name a decimal number? It doesn't seem like a good thing for the compiler to accept.
Please disregard my comment about where the quotes are in DEFINED("EXCLUDE-352729.2") vs DEFINED(EXCLUDE-"352729.2"). I understand that is AppBuilder generated.
Unfortunately I can't
Unfortunately I can't comment on why it's a decimal number. I can fix it in my code base. I think it's the only example, but as ABL allows it would expect it to be handled.
So, we have two things here.
So, we have two things here.
The first, there is no real sample for because there is just one line that throws an unexpected token error. One would guess this is unsupported syntax in the current version.
The second is the function definition with global defines. In the wild, it is in nested includes, so I have asked James to see if he can replicate it in an isolated .p or whether it is dependent on the nesting.
You can send me a zip file
You can send me a zip file full of include files - that's fine. Even a .df for that matter. As long as I can have something I can feed into COMPILE without error.
BTW, I have modified my test
BTW, I have modified my test scanner to include an optional tree walk so that it doesn't just create the parse tree, but also navigates it. But, it isn't doing anything with the node other than the child and sibling links. Anything else you would put in there to help validate it.
What puzzles me is that something like the unexpected token error seems like something that would have to be happening in the initial parse, i.e., in treeParser01(). So, if that is going to blow up as a part of ABL2DB, it should blow up in TreeScanner as well. No?
It now appears that
It now appears that TestScanner wasn't really doing what we thought it was, but this seems to have been fixed and it is now throwing errors. This should make it easier to isolate missing keywords and the like.
Right, it is just with
Right, it is just with multiple includes and multiple layers of nesting, even identifying all the code needed is a bit messy. But, James is working on getting something together.
I've managed to isolate it
I've managed to isolate it within my application but the nested includes are a complete nightmare scenario. I've manage to get a single file that fails and I'm pairing it down a block at a time until it works ok.
Here's another one from
Here's another one from James:
Right I've got it as narrow as I can but it's a right pain.
EnergyTradingDetails.p includes:
{saa-status-descid.i}
That includes
{saa-status.i &INC-SAA-TABLE = lb-Inc-SAA-StatusDescID}
That then looks like this:
&IF DEFINED(INC-TIS-TABLE) EQ 0 &THEN
&GLOBAL-DEFINE INC-TIS-TABLE {&TIS-TABLE}
&ENDIF
&IF DEFINED(INC-SAA-TABLE) EQ 0 &THEN
&GLOBAL-DEFINE INC-SAA-TABLE {&SAA-TABLE}
&ENDIF
&IF DEFINED(INC-TIS-TABLE-TIS-TEA-MODE) EQ 0 &THEN
&GLOBAL-DEFINE INC-TIS-TABLE-TIS-TEA-MODE {&INC-TIS-TABLE}.TisTeaMode
&ENDIF
FUNCTION fn-SAA-Complete RETURNS LOGICAL
(BUFFER ib-TIS FOR {&INC-TIS-TABLE},
BUFFER ib-SAA FOR {&INC-SAA-TABLE}) :
END FUNCTION.
And it seems to be the function definition that's bombing out. But it doesn't bomb out if I replace the preprocessors for the buffer names with real tables.
Really struggling to
Really struggling to reproduce this example. The code itself is not critical so I'm removing it for now.
I have a different example. Although I have no idea AT ALL how it's meant to work...
PROCEDURE UsageMonthProfile :
DEFINE INPUT PARAMETER ip-CtpApplicationMeaningType LIKE INPUT ICMAS.ChargeType.CtpApplicationMeaningType NO-UNDO.
END.
[15/10/02@22:34:23.113+0100] P-006176 T-004132 1 4GL APPL ScanOneCU: Other error in C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p: org.prorefactor.refactor.RefactorException: Could not find input field ICMAS.ChargeType.CtpApplicationMeaningType C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p:10396 -> File: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p Line: 10396 Column: 10 Type: INPUT Text: INPUT
[15/10/02@22:34:23.113+0100] P-006176 T-004132 1 4GL OtherERROR Other error (cont.): org.prorefactor.core.PRCException: Could not find input field ICMAS.ChargeType.CtpApplicationMeaningType C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p:10396 -> File: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p Line: 10396 Column: 10 Type: INPUT Text: INPUT
[15/10/02@22:34:23.113+0100] P-006176 T-004132 1 4GL OtherERROR Other error (cont.): java.lang.Error: Could not find input field ICMAS.ChargeType.CtpApplicationMeaningType C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\savmgrpm.p:10396
[15/10/02@22:34:23.115+0100] P-006176 T-004132 1 4GL CALLSTACK DoParse com.cintegrity.Proparse.ScanOneCU at line 80 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\ScanOneCU.r)
[15/10/02@22:34:23.119+0100] P-006176 T-004132 1 4GL CALLSTACK ScanOne com.cintegrity.Proparse.TestScanner at line 158 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:34:23.123+0100] P-006176 T-004132 1 4GL CALLSTACK ScanAll com.cintegrity.Proparse.TestScanner at line 79 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:34:23.125+0100] P-006176 T-004132 1 4GL CALLSTACK Process com.cintegrity.Proparse.TestScanner at line 72 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:34:23.132+0100] P-006176 T-004132 1 4GL CALLSTACK C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\TestScanner.p at line 85 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\TestScanner.r)
And another one: &IF
And another one:
&IF DEFINED(EXCLUDE-"352729.2") = 0 &THEN
&ANALYZE-SUSPEND _UIB-CODE-BLOCK _PROCEDURE "352729.2" Procedure
PROCEDURE 352729.2:
END PROCEDURE.
Standard AppBuilder code.
[15/10/02@22:35:59.653+0100] P-006176 T-004132 1 4GL APPL ScanOneCU: Other error in C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\stdjnlxp.p: org.prorefactor.refactor.RefactorException: java.lang.RuntimeException: java.lang.IllegalArgumentException: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\stdjnlxp.p:146 Bad DEFINED function in &IF preprocessor condition
[15/10/02@22:35:59.653+0100] P-006176 T-004132 1 4GL OtherERROR Other error (cont.): java.lang.RuntimeException: java.lang.IllegalArgumentException: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\stdjnlxp.p:146 Bad DEFINED function in &IF preprocessor condition
[15/10/02@22:35:59.653+0100] P-006176 T-004132 1 4GL OtherERROR Other error (cont.): java.lang.IllegalArgumentException: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\stdjnlxp.p:146 Bad DEFINED function in &IF preprocessor condition
[15/10/02@22:35:59.654+0100] P-006176 T-004132 1 4GL CALLSTACK DoParse com.cintegrity.Proparse.ScanOneCU at line 80 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\ScanOneCU.r)
[15/10/02@22:35:59.656+0100] P-006176 T-004132 1 4GL CALLSTACK ScanOne com.cintegrity.Proparse.TestScanner at line 158 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:35:59.657+0100] P-006176 T-004132 1 4GL CALLSTACK ScanAll com.cintegrity.Proparse.TestScanner at line 79 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:35:59.658+0100] P-006176 T-004132 1 4GL CALLSTACK Process com.cintegrity.Proparse.TestScanner at line 72 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\com\cintegrity\Proparse\TestScanner.r)
[15/10/02@22:35:59.660+0100] P-006176 T-004132 1 4GL CALLSTACK C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\TestScanner.p at line 85 (C:\Users\jpalmer\Copy\ABL2DB Workspace\ABL2DB\src\TestScanner.r)
One report from James: Ok
One report from James:
Ok first problem found. And it's a proparse issue.
[15/10/02@16:47:26.449+0100] P-005052 T-008124 1 4GL APPL Other error in C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\edimgrpm.p line 0: org.prorefactor.refactor.RefactorException: C:\Users\jpalmer\Copy\ABL2DB\ICMAS\PCODE\edimgrpm.p:3593:5: unexpected token: UPDATE
[15/10/02@16:47:27.023+0100] P-005052 T-008124 1 4GL OtherERROR Other error (cont.): antlr.NoViableAltException: NoViableAlt
The offending code is
SYSTEM-DIALOG GET-DIR lv-FileName
INITIAL-DIR ip-InitialDir
TITLE ip-DescId
UPDATE lv-ok.
Does this indicate a need for a Proparse extension?
Re: Parsing test
I'd be happy to look at it, but first I must have a sample which reproduces it.
Identifying that sample is
Identifying that sample is one of the goals of the TestScanner.cls. I was thinking of adding an option to it to walk the tree to see if that would reveal the error.
James thinks he may have isolated a sample ... if that proves out, I will get him to give you a copy and more specifics on what is blowing up. I think he is running it debug.
I will attach the class which does the parsing to see if anything there stands out for you.
NPE
Do you have (or can you construct) a smallish sample that reproduces this?
It is not being simple. The
It is not being simple. The programs which are throwing the errors are full of nested includes and the like. If I can get it to happen outside the context of ABL2DB, we should be able to provide a test case more easily ... that was the point of the TestScanner.cls, except it isn't throwing any errors.