Look sharp

Fluent API or Fluent Interfaces
https://www.primaryobjects.com/2011/02/07/fluent-interfaces-in-c-net-with-expression-builder-method-chaining-and-rpg-games/

Fluent XML document validation 

Fluent Interface

An interface designed in a way that actions are chained together gives a fluent, readable and natural style of logic
https://en.wikipedia.org/wiki/Fluent_interface

• It improves code readability and flow, like writing sentences
• It is commonly used in query builders, configuration settings, and ORMs (like Entity Framework).

In this article you will have a POC (Proof of Concept) which uses a fluent interface for the steps needed to fully validate an XML document
A fluent interface fits perfectly with the step you need to go though in a specific ordering when you validate XML data

Interfaces we are going to use

• Create Document
  
  • Create prepares the document for validation
• Check syntax
  • First check document for syntax errors
• Check structure
  • Next check the document for structural integrity, eg do schema validation
• Check content
  • Basic validation is done we can check the document for the business requirements
• Transform document
  • Or we are ready to style and transform the document to a readable presentation by using eg a stylesheet 
• Result
  • Return the validation result

Main class to stich it all together

• XMLValidator
  
  • Inherits
    • Document 
      • Create
    • ValidateSyntax
      • CheckSyntax
    • ValidateStructure
      • CheckStructure
    • Validated
      • CheckContent
      • TransformContent
    • ValidationResult
      • GetMessage (Result)

The resulting work flow

• Fulfilment 
  1. Validation occurs in strict order or chain defined by the returned interface in the methods called
  2. Response is added to StringBuilder by each step
  3. Validation result is returned to user and thus the chain is terminated

POC of FluentValidator

All interfaces

Steps needed for the document object to be prober validated

// Interfaces used for building the fluent validator	
namespace FluentValidator.Interface
{
	/// <summary>
	/// Prepare document for validation
	/// </summary>
	public interface IDocument
	{
		public IValidateSyntax Create();
	}

	/// <summary>
	/// RuUn a syntax check on the document
	/// </summary>
	public interface IValidateSyntax
	{
		public IValidateStructure CheckSyntax();
	}

	/// <summary>
	///	Validate the structure of the document using a schema
	/// </summary>
	public interface IValidateStructure
	{
		public IValidated CheckStructure();
	}

	/// <summary>
	/// Document is structural and semantic valid
	/// </summary>
	public interface IValidated
	{
		public IValidationResult CheckContent();
		public IValidationResult TransformContent();
	}

	/// <summary>
	/// Return validation result
	/// </summary>
	public interface IValidationResult
	{
		public string GetMessage();
	}
}

The concrete class implementation using the interfaces we have created

using System.Text;
using FluentValidator.Interface;

namespace FluentValidator.Validator
{
	/// <summary>
	/// Concrete implementation of the fluent validator
	/// </summary>
	/// <seealso cref="FluentValidator.Interface.IValidateSyntax" />
	/// <seealso cref="FluentValidator.Interface.IValidateStructure" />
	/// <seealso cref="FluentValidator.Interface.IValidated" />
	/// <seealso cref="FluentValidator.Interface.IValidationResult" />
	/// <seealso cref="FluentValidator.Interface.IDocument" />
	public class ValidateXML : IValidateSyntax, IValidateStructure, IValidated, IValidationResult, IDocument
    {
        private StringBuilder message = new StringBuilder();

		/// <summary>
		/// Creates the instance.
		/// </summary>
		/// <returns></returns>
		public IValidateSyntax Create()
        {
            return new ValidateXML();
        }
		/// <summary>
		/// Checks the syntax.
		/// </summary>
		/// <returns>IValidateStructure</returns>
		public IValidateStructure CheckSyntax()
        {
            message.Append("Syntax validated\r\n");
            return this;
        }

		/// <summary>
		/// Checks the structure.
		/// </summary>
		/// <returns>IValidated</returns>
		public IValidated CheckStructure()
        {
            message.Append("Structure validated\r\n");
            return this;
        }

		/// <summary>
		/// Checks the content.
		/// </summary>
		/// <returns>IValidationResult</returns>
		public IValidationResult CheckContent()
        {
            message.Append("Content validated\r\n");
            return this;
        }

		/// <summary>
		/// Transforms the content.
		/// </summary>
		/// <returns>IValidationResult</returns>
		public IValidationResult TransformContent()
		{
			message.Append("Stylesheet applied\r\n");
			return this;
		}

		/// <summary>
		/// Gets the message.
		/// </summary>
		/// <returns>string</returns>
		public string GetMessage()
        {
            return message.ToString();
        }

    }
}

POC program to run

using FluentValidator.Validator;

namespace FluentValidator
{
    internal class Program
    {
        static void Main(string[] args)
        {
            ValidateXML validate = new ValidateXML();

			Console.WriteLine("Validate XML file:");
			Console.WriteLine("------------------");

			Console.WriteLine(
                validate.Create()
                    .CheckSyntax()
                    .CheckStructure()
                    .CheckContent()
                    .GetMessage()
                );

			Console.WriteLine("");
			Console.WriteLine("Validate and transform XML file:");
			Console.WriteLine("--------------------------------");

			Console.WriteLine(
				validate.Create()
					.CheckSyntax()
					.CheckStructure()
					.TransformContent()
					.GetMessage()
				);

			Console.ReadKey();
        }
    }
}

Console output from the program

Validate XML file:
------------------
Syntax validated
Structure validated
Content validated

Validate and transform XML file:
--------------------------------
Syntax validated
Structure validated
Stylesheet applied

Try the example at Fiddle

https://dotnetfiddle.net/gqNieE

For mappings created in the Business Integration Converter (BIC) at times you need local or global procedures following the DRY principles in coding (Don't repeat yourself)
Maybe you need to enhance the BIC script by adding some inline Java code like shown in eg the unescapeHTML  procedure found on these pages

It can be parameters in procedure signature or local variables, in the unescapeHTML procedure we can find this code

// Procedure parameter used in java code
String source = _StrVar_PSOURCE.getString();

You can find a lot of examples of this style of Java code in your build folders, in <source>\<project>\build\test\com\seeburger\jucon\mapping you can examine the Java files from your recent build

An example

// Java code example assigning an IDOC numer to a numeric variable
_NumVar_IDOC_NUMBER.setNum(stdin.getField("THIS:DOCNUM"));

Another small example which uses Java variables described is the getUUID method

You can find some examples on this site: Methods using inline Java

An utility to add leading zeros up to given length to a numeric string 
Please note Alphanumeric string will be left untouched

A procedure to use in Seeburger BIC Mapping Designer
It will extend your library for string handling

/*  -----------------------------
    Peter Lykkegaard, 22 mar 2019
    -----------------------------   
	Method
	   zeroFill
	Parameters
	   value$: String to prefix with zeros
	   Num%: Number of characters, length of string after prefixzing
	Returns
	   $
	Description
	   If the parameter value is a numeric value and 
	   the length of the string does not exceed length of the value
   */

   if (isNumeric(value$) && len(value$) < num%)
       copy lfill(value$, "0", num%) to value$;
   endif

   exitProc(value$);

Split text on right most space before limit is reached
Use to split text on several lines in receiving document, eg edifact LIN-IMD-C273 has several 7008 text element

//    -----------------------------
//    Peter Lykkegaard, 22 mar 2019
//    -----------------------------
//    Split long note on right most space before limit is reached
//    TODO! Check parameters if they are of valid type/content
//
//    Name: splitNoteBySpace
//    Description: Method to split a given string by latest space before length limit
//    Parameters
//        input$, alphanumerical / String which needs to be split
//        max$, alphanumerical / Max characters on each line
//    Output, alphanumeric / array / Array of strings to return
 
local lvOutput$[];

#importJavaStart
    import java.util.regex.Matcher;
    import java.util.regex.Pattern;
#importJavaEnd
 
#javastart

    String input = _StrVar_PINPUT.getString();
    int max = (int)_NumVar_PMAX.getNum();
    int len = input.length();
    
    String regex = "(.{1," + max + "})(?:\\s|$)";
 
    Pattern pattern = Pattern.compile(regex, Pattern.MULTILINE);
    Matcher matcher = pattern.matcher(input);
    
    _StrVar_LVOUTPUT.getJuVar(0, 2).setString(input);

    if (len > max)
    {
        int idx = 0;     // Entries in output   
        while (matcher.find()) {
            for (int i = 1; i <= matcher.groupCount(); i++) {
                _StrVar_LVOUTPUT.getJuVar(idx, 2).setString(matcher.group(i).toString());
            }
            idx++;
        }
    }
#javaEnd
     
exitProc(lvOutput$);

Validate EDI (XML, Edifact and Ansi X12)  

Site

• EDI and Content File Validator - validatefile.dk

Functionality

• Multiupload
  • No data is stored on the server
  • MIME type is detected based on file content
    • It is possible to upload without file extensiom
    • All other files than text (edifact / ansi x12) or xml are skipped / not supported
• Validate structure (eg schema validation)
• Validate business logic (eg schematron validation)
• XSLT Styling, readable document in HTML
• Calculate content and compare with values in eg an invoice

Milestones

• Multiupload based on Dropzone JS: https://www.dropzone.dev/
• Implemented Mime-Detective in class factory: https://github.com/MediatedCommunications/Mime-Detective
• Prepared and created concept to post files several times from Dropzone JS queue
  1. Upload to local client (browser) 
  2. Send file content to server to either
     1. Check and validate content
     2. Check business logic
     3. Style document
     4. Calculate prices and compare with document values
• Use RaspConfiguration (Converted to JSON)
  • Detect document type to prepare for XSD (logic) and Schematron (business) validation 
  • LINQ lookup to get correct DocumentTypeConfig
  • Schema validation with PSVI
    • https://www.w3.org/XML/2002/05/psvi-use-cases
• Cookie consent
• AntiforgeyTokens
  • Antiforgery: https://learn.microsoft.com/en-us/aspnet/web-api/overview/security/preventing-cross-site-request-forgery-csrf-attacks
  • JSON: https://stackoverflow.com/questions/32287901/how-do-i-use-antiforgerytoken-with-dropzone-js-and-mvc-5-with-vanilla-js
• NuGet package: MimeDetective (OK)
• Mime Mapping (Done): https://www.garykessler.net/library/file_sigs.html
• Alternative API content check: https://www.appsloveworld.com/csharp/100/139/alternative-to-findmimefromdata-method-in-urlmon-dll-one-which-has-more-mime-type (Skipped)
• When XML Reader is opened reset the stream to the starting point: stream.Seek(0, SeekOrigin.Begin);
• OIOXML schemas, validation and styling on Gitlab: https://rep.erst.dk/git/openebusiness/library/java/-/tree/6843728b8e72108eeb1dc2a30ed324fb61cb4c2b/dk.gov.oiosi.examples.client/resources
• Display XML content on web page
• Verification test on Kestrel and local IIS

Current task

• Scoped the project, only a few XML and EDIFACT versions will be implemented
  • OIOUBL (danish adaption of the UBL standard)
  • Peppol BIS
  • Standard eg EDIFACT D96 are considere in a limited version as a start as a proof of concept
  • cXML and xCBL are a possibility
• Store validation classes / objects in memory cache to use on subsequent requests
• XSLT transformations uses SaxonHE
  • All stylesheet versions are supported
  • Included stylesheets or XML documents are supported
• Documentation of source code (XML documentation)
  • SandCastle is used to generate HTML or MD documents
  • Add content documentation to source code

Release

•  WIP
  • Jun 23: Only local test / development atm, date for a first beta release not estimated
  • Oct 23: Slow process with local test / development, getting closer to a beta release 
  • Dec 23: Testing, prepare for release on VPS with Ubuntu
  • Jan 14: Project running on VirtualBox / Ubuntu 23.10 / NET 8
    • NGINX as Reverse Proxy in fron of Kestrel web server
  • Jan 14: Test memory usage to validate 8 Gb mem limit
  • Jan 23: Released on https://www.validatefile.dk
    • VPS with 2 CPU, 8 Gb mem running Ubuntu 23.10 server and .NET 8
  • Feb 07: Small adjustment, preparation for xCBL and cXML validation, About page added

To do list for project: 

• Create Edifact Icon in SVG
  • Tools needed:
    • Inkscape SVG editor
    • Fontforge: Create WOF2 package
• Validate file content
  • javascript:
    • https://www.dotnetexpertguide.com/2011/05/validate-uploaded-image-content-in.html
    • https://gist.github.com/aziz-blr/fdef65aed6d60729b47f1f52bd6bd0af
  • XML schema validation: https://learn.microsoft.com/en-us/dotnet/api/system.xml.schema.extensions.getschemainfo?view=net-7.0
    • Will be replaced with SaxonHE libraries
• Additional upload for schematron valditation, styling and calculation:
  • https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/Ajax-JavaScript-file-upload-example
• Document examples from Pagero:  
  • https://rep.erst.dk/git/openebusiness/common/-/tree/master/resources/Documents?ref_type=heads
  • https://pagero.github.io/business-documents/invoice-creditnote/
  • https://github.com/pagero/integration-resources/tree/main/example-files
• Document styling
  • Idea for future output from html stylresheets
    https://github.com/unimaze/unimaze-peppol-stylesheets?tab=readme-ov-file
• Display XML content on web page: (Done)
  • https://learn.microsoft.com/en-us/previous-versions/aspnet/13ftcwy9(v=vs.100)
  • https://blorgh.wordpress.com/2006/10/17/displaying-xml-in-the-net-webbrowser-control/
  • https://learn.microsoft.com/en-us/answers/questions/1181957/show-xml-in-webbrowser-control
  • Use ContentResult in Controller, open new window in JS and paste html content (POC done)
• Edifact example source / projects
  • https://github.com/8/Edifact
  • https://github.com/indice-co/EDI.Net
  • https://www.nuget.org/packages/indice.Edi (same as above)
  • Extras I need to check later: https://michaelachrisco.github.io/Electronic-Interchange-Github-Resources
  • Former EDIFabric: https://github.com/EdiWeave/EdiWeave
    • Characterset / Encoding:
      https://blog.sandro-pereira.com/2009/08/15/edifact-encoding-edi-character-set-support/
• favicon (Done)
  • https://stackoverflow.com/questions/4014823/does-a-favicon-have-to-be-32%C3%9732-or-16%C3%9716
• Popup window with validation messages (Abandoned)
  • https://stackoverflow.com/questions/1216114/how-can-i-make-a-div-stick-to-the-top-of-the-screen-once-its-been-scrolled-to
• Milliseconds elapsed (Done)
  • https://stackoverflow.com/questions/41632942/how-to-measure-time-elapsed-on-javascript

Sandcastle

• https://www.red-gate.com/simple-talk/development/dotnet-development/taming-sandcastle-a-net-programmers-guide-to-documenting-your-code/

Use local postman for testing?

• https://weblog.west-wind.com/posts/2017/Sep/14/Accepting-Raw-Request-Body-Content-in-ASPNET-Core-API-Controllers?Page=2
  Working, use vaiable name from controller as property name in postman

Unit test

• https://www.toptal.com/qa/how-to-write-testable-code-and-why-it-matters

Check for vulnerability

• https://pvs-studio.com/en/
• https://itnext.io/vulnerabilities-due-to-xml-files-processing-xxe-in-c-applications-in-theory-and-in-practice-f94912216093

GDPR (mostly done)

• Cookie consent: https://zetbit.tech/categories/asp-dot-net-core/10/creating-a-cookie-policy-consent-popup-in-asp-net-6
• (GDPR) support in ASP.NET Core: https://learn.microsoft.com/en-us/aspnet/core/security/gdpr?view=aspnetcore-7.0

Information or questions: Please contact plykkegaard at gmail dot com