Source to requirement tracing with Structured Text (PLCs)

[xw-mk]

I'm working on a project involving a PLC that uses ST (Structured Text) for the majority of code.

I'm trying to get the source code tracing working and not having much luck. Strictdoc is finding the .st files and getting the correct line count, but not identifying markers. An example .st file:

FUNCTION_BLOCK FB_Test
VAR_INPUT
	i_xA: BOOL;
	i_xB: BOOL;
END_VAR
VAR_OUTPUT
	o_xX: BOO;
END_VAR

// --- BEGIN IMPLEMENTATION ---
// @relation(SF_REQ-1, scope=range_start)
IF i_xA AND i_xB THEN
	o_xX := TRUE;
ELSE
	o_xX := FALSE;
END_IF
// @relation(SF_REQ-1, scope=range_end)

I've tested with a python file in the same project and that works without issue. I've renamed the .st file to .c and then the markers are found.

I know there's not direct support for Structured Text files but Is there something obvious I'm missing, or a simple workaround? I will also need to parse .xml files in the future which contain snippets of ST code. I guess I'm looking for a generic way to identify markers in test based files of an unknown type.

I am also open to helping add support for .st myself but would need some guidance on where to start looking in the codebase.

Thanks!

[stanislaw]

Hey @xw-mk,

thanks a lot for your feedback and request.

I have added reading ST with an integration test fixture and it is working on my end as expected: #2570. The only real change I had to make was to disable the syntax highlighting (.st files were incorrectly recognized as Smalltalk files), see here: https://github.com/strictdoc-project/strictdoc/pull/2570/files#diff-6ae2711c07efb5292a7f10fd2298e57329d1f3165b348d094514f639224ea71eR178.

Please review the test fixture setup and double-check if you may be missing any configuration detail, for example, that your config looks similar to what I have in the PR: https://github.com/strictdoc-project/strictdoc/pull/2570/files#diff-540085da07ff4499ac1e9f2759edca28a0b103121a7d1e7e33aad49cece20c9eR1.

My fix with the syntax highlighting has not been released yet, so I suggest that you install the nightly version of StrictDoc to not get confused by the incorrect Smalltalk syntax highlighting for *.st files:

pip install -U --pre git+https://github.com/strictdoc-project/strictdoc.git@main

This is how it looks like:

---

Regarding the syntax highlighting: it looks like someone created a Lexer for this ST language but it has never been merged to the Pygments repo: pygments/pygments#1002 (comment).

We could enable the syntax highlighting by suggesting @robertobernabe who is the author of that GitHub gist Pygments Lexer for IEC61131 3 Structured Text to open a PR to Pygments. (Preferred and cleaner approach).

cc @robertobernabe: I hope I can reach you here by tagging your name: would you be open to creating a Pull Request with your IEC61131 ST Lexer to the official Pygments repository?

[robertobernabe]

@stanislaw Hi, yes indeed I would be open to create a PR to Pygments. Let's see when I have time for doing that. Is it urgent?
I could open up a PR next week, maybe.

[robertobernabe]

@stanislaw Had some time in the morning and created already the corresponding PR here: pygments/pygments#2993.

[stanislaw]

@robertobernabe this is great! Thanks for taking the time. Let's see how quickly this gets merged. The last update on pygments seems to be 3 months ago 🤔🤔🤔

[xw-mk]

@stanislaw firstly, thanks for the fast turnaround!

I've tried the nightly build and am still having issues. I've compared the example in your test fixture with our requirements and after some back and forth I believe the source of the issue is that we have requirements with custom prefixes (SF_REQ instead of REQ for example). This doesn't appear to be an issue with .c or .py files, but for anything else (ie .st files) there are some peculiarities.

Source code

With these requirements:

[DOCUMENT]
TITLE: Example: Traceability between requirements and source files

[REQUIREMENT]
UID: REQ-001
TITLE: Requirement to source reference
STATEMENT: This requirement references the whole file from requirements.
RELATIONS:
- TYPE: File
  VALUE: file.st

[REQUIREMENT]
UID: REQ-002
TITLE: Source to requirement range reference
STATEMENT: This requirement references a range in the file.

[REQUIREMENT]
UID: REQ-003
TITLE: Requirement to source range reference
STATEMENT: This requirement references a range in the file from the requirement.
RELATIONS:
- TYPE: File
  VALUE: file.st
  LINE_RANGE: 2, 4

[REQUIREMENT]
UID: SF_REQ-001
TITLE: Custom requirement prefix range reference
STATEMENT: This requirement has a custom prefix, and is referenced in the file.

And this source file:

// @relation(SF_REQ-001, scope=file)
FUNCTION_BLOCK FB_Test
VAR_INPUT
	i_xA: BOOL;
	i_xB: BOOL;
END_VAR
VAR_OUTPUT
	o_xX: BOOL;
END_VAR

// --- BEGIN IMPLEMENTATION ---
// @relation(REQ-002, scope=range_start)
IF i_xA AND i_xB THEN
	o_xX := TRUE;
ELSE
	o_xX := FALSE;
END_IF
// @relation(REQ-002, scope=range_end)

Running strictdoc with the file above I get this:

There are 2 things going wrong here:

1. SF_REQ-001 relation in the source file is not detected.
2. The REQ-002 range relation in the source file is falsely attributed to REQ-001.

Appreciate the support, let me know if you need any additional info.

[stanislaw]

@xw-mk I have just merged this where I:

• Fixed the UID regex to recognize the underscores — that was the reason why the SF_REQ-001 relation was not detected.
• Could not reproduce the REQ-002. I created a test fixture using your example and it is working correctly, see it in the PR.

Please install the Pip from the last main branch and let me know how it works now.

[xw-mk]

@stanislaw I've just tried with the latest build and all looks good. SF_REQ's are being found correctly and I can't replicate the REQ-002 issue seen above so perhaps that has been fixed with the regex changes.

Thanks for the fast turnaround, and appreciate the hard work that's gone into Strictdoc!