Glasswall Runner

Overview

The Glasswall Runner is a wrapper around the Glasswall engine using the Glasswall Python Wrapper. It features several modes which can be used to process or transform files or directories of files.

Command-line Interface

-h, --help            show this help message and exit
-m {Analysis,AnalysisProtect,Combine,CombinePerformanceTest,ExportUnpackCombine,Export,Import,Pack,Protect,Redact,Uncombine,Unpack,UncombinePackImportProtect,UnpackProtectPack}
                      Glasswall Runner functionality to call.
-i INPUT              Absolute path to input file or directory containing files to process.
-o OUTPUT             Absolute path to output file or directory to write processed files.
-l LIBRARIES_PATH     Absolute path to directory containing Glasswall libraries.
-c CONFIG             Optional absolute path to content management policy config XML file.
--quarantine QUARANTINE
                      Optional absolute path to directory to move files that Glasswall failed to process (unsupported or nonconforming files).
--raise-unsupported   Terminate early when Glasswall cannot process a file.
--logging {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
                      Set Glasswall Runner logging level.
--logging-glasswall {CRITICAL,ERROR,WARNING,INFO,DEBUG,NOTSET}
                      Set Glasswall Python Wrapper logging level.
--debug               Enable debug mode. In debug mode, quarantine will copy instead of move files.
--homoglyphs HOMOGLYPHS
                      Optional absolute path to homoglyphs JSON file (Redact mode).
--redact-list REDACT_LIST
                      Optional absolute path to a .txt file that contains search terms to redact. One search term per line (Redact mode).
--output-report OUTPUT_REPORT
                      Optional absolute path to output file or directory to write reports (ExportArchive/ImportArchive/Redact mode).
--archive-whitelist ARCHIVE_WHITELIST
                      Dangerous. Optional absolute path to a .txt file that contains whitelisted file formats that are never removed from archives, even if sanitisation fails. One file format per line (UnpackProtectPack mode).
--upp-report UPP_REPORT
                      Optional absolute path to output file to write JSON report (UnpackProtectPack mode).
--archive-file-type {7z,bz2,gz,tar,xz,zip}
                      Optional archive file format to use when packing with ArchiveManager, default: zip (Pack/UncombinePackImportProtect mode).
--unpack-include-file-type
                      Optional, include file type in directory name when unpacking archives, default: false (Unpack mode).
--performance-test PERFORMANCE_TEST
                      Optional absolute path to output file to write JSON performance test report (Analysis/AnalysisProtect/Export/Import/Protect mode).
--combine-performance-test-filter COMBINE_PERFORMANCE_TEST_FILTER
                      Only match files ending with the filter, default '.pt.json'. (CombinePerformanceTest mode).

Configuration

A file path to a content management policy config XML file can be specified using the optional -c command line argument. The Glasswall engine uses this config to decide which types of content within files to allow, sanitise, or disallow. If this argument is not set then the default "sanitise all" content management policy will be applied.

<?xml version="1.0" encoding="utf-8"?>
<config>
    <gifConfig>
        <metadata>sanitise</metadata>
    </gifConfig>
    <pdfConfig>
        <acroform>sanitise</acroform>
        <actions_all>sanitise</actions_all>
        <digital_signatures>sanitise</digital_signatures>
        <embedded_files>sanitise</embedded_files>
        <embedded_images>sanitise</embedded_images>
        <external_hyperlinks>sanitise</external_hyperlinks>
        <internal_hyperlinks>sanitise</internal_hyperlinks>
        <javascript>sanitise</javascript>
        <metadata>sanitise</metadata>
    </pdfConfig>
    <pptConfig>
        <embedded_files>sanitise</embedded_files>
        <embedded_images>sanitise</embedded_images>
        <external_hyperlinks>sanitise</external_hyperlinks>
        <internal_hyperlinks>sanitise</internal_hyperlinks>
        <javascript>sanitise</javascript>
        <macros>sanitise</macros>
        <metadata>sanitise</metadata>
        <review_comments>sanitise</review_comments>
    </pptConfig>
    <sysConfig>
        <export_embedded_images>true</export_embedded_images>
        <interchange_best_compression>false</interchange_best_compression>
        <interchange_pretty>false</interchange_pretty>
        <interchange_type>sisl</interchange_type>
        <run_mode>enablerebuild</run_mode>
    </sysConfig>
    <tiffConfig>
        <geotiff>sanitise</geotiff>
    </tiffConfig>
    <wordConfig>
        <dynamic_data_exchange>sanitise</dynamic_data_exchange>
        <embedded_files>sanitise</embedded_files>
        <embedded_images>sanitise</embedded_images>
        <external_hyperlinks>sanitise</external_hyperlinks>
        <internal_hyperlinks>sanitise</internal_hyperlinks>
        <macros>sanitise</macros>
        <metadata>sanitise</metadata>
        <review_comments>sanitise</review_comments>
    </wordConfig>
    <xlsConfig>
        <dynamic_data_exchange>sanitise</dynamic_data_exchange>
        <embedded_files>sanitise</embedded_files>
        <embedded_images>sanitise</embedded_images>
        <external_hyperlinks>sanitise</external_hyperlinks>
        <internal_hyperlinks>sanitise</internal_hyperlinks>
        <macros>sanitise</macros>
        <metadata>sanitise</metadata>
        <review_comments>sanitise</review_comments>
    </xlsConfig>
</config>

Note: Some modes may require sysConfig switches to be set to specific nondefault values in order to function, such as Redact mode requiring the interchange_type switch to be set to xml. When using a custom content management policy the Glasswall Runner will prioritise using the required sysConfig switch value.

Modes

A mode must be specified when calling the Glasswall Runner using the -m command line argument. Different modes have different required and optional command line arguments. Some modes function without using the Glasswall engine and do not require a -l argument. Available modes are listed below and are case sensitive.

Protect

Required args: m, i, o, l

Protect a single file.

glasswall_runner.exe -m Protect -i "C:\input\DOCXimages.docx" -o "C:\output\protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"

Protect all files in a directory.

glasswall_runner.exe -m Protect -i "C:\input" -o "C:\output\protect_directory" -l "C:\azure\current\2023-06-05"

Protect all files in a directory using a custom content management policy, moving nonconforming, unsupported, and disallowed files to a quarantine directory.

glasswall_runner.exe -m Protect -i "C:\input\some_nonconforming" -o "C:\output\protect_directory2" -l "C:\azure\current\2023-06-05" -c "C:\input\config_docx_disallow_embedded_images.xml" --quarantine "C:\output\protect_directory2_quarantine"

<?xml version="1.0" encoding="utf-8"?>
<config>
    <wordConfig>
        <embedded_images>disallow</embedded_images>
    </wordConfig>
</config>

Protect all files in a directory, saving performance test metrics.

glasswall_runner.exe -m Protect -i "C:\input" -o "C:\output\protect_directory_performance_test" -l "C:\azure\current\2023-06-05" --performance-test "C:\output\protect_directory_performance_test\input.pt.json"

[
    {
        "input_file": "C:/input/DOCXimages.docx",
        "elapsed_milliseconds": 636,
        "input_sha256": "0c33844c15589033ab7586c4ddc6378ff19a35186d72848c1562dedd2b9f9297",
        "mode": "Protect",
        "passed": true
    },
    {
        "input_file": "C:/input/config.xml",
        "elapsed_milliseconds": 21,
        "input_sha256": "bd34129adc72ea3b8af818026f40229908abe795b849b9f0197d7630c862c2f3",
        "mode": "Protect",
        "passed": false
    },
    {
        "input_file": "C:/input/config_docx_disallow_embedded_images.xml",
        "elapsed_milliseconds": 9,
        "input_sha256": "dc0f758eb72cc778d9b6693a18204db5594b092ccc1bf837ea1ae010bca694e9",
        "mode": "Protect",
        "passed": false
    },
    {
        "input_file": "C:/input/config_redact.xml",
        "elapsed_milliseconds": 8,
        "input_sha256": "96693f0fc59ae305f2ab782cf2243f771d2a1ea68d7832be1af31b2fc9370c78",
        "mode": "Protect",
        "passed": false
    },
    {
        "input_file": "C:/input/redact_list.txt",
        "elapsed_milliseconds": 5,
        "input_sha256": "a12927c25a413290032c8fe7aabe30a20468d6984012014790a631ca3e2c00c4",
        "mode": "Protect",
        "passed": false
    },
    {
        "input_file": "C:/input/upp/archive_whitelist.txt",
        "elapsed_milliseconds": 6,
        "input_sha256": "e9aafe226374d2f75e719d4b9b5f4153b151841e17297410af78c928ff02e4b6",
        "mode": "Protect",
        "passed": false
    },
    {
        "input_file": "C:/input/upp/nested2.zip",
        "elapsed_milliseconds": 46,
        "input_sha256": "dfa3d58ccb65e2128dfe9ff62a934b5fb4731f431c1f72f0820fbf0fa480bbaa",
        "mode": "Protect",
        "passed": false
    }
]

Analysis

Required args: m, i, o, l

Analyse a single file.

glasswall_runner.exe -m Analysis -i "C:\input\DOCXimages.docx" -o "C:\output\analysis\DOCXimages.docx.xml" -l "C:\azure\current\2023-06-05"

Analyse all files in a directory.

glasswall_runner.exe -m Analysis -i "C:\input" -o "C:\output\analysis_directory" -l "C:\azure\current\2023-06-05"

AnalysisProtect

Required args: m, i, o, l

Analyse and Protect a single file.

glasswall_runner.exe -m AnalysisProtect -i "C:\input\DOCXimages.docx" -o "C:\output\analysis_protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"

Analyse and Protect all files in a directory.

glasswall_runner.exe -m AnalysisProtect -i "C:\input" -o "C:\output\analysis__protect_directory" -l "C:\azure\current\2023-06-05"

Export

Required args: m, i, o, l

Export a single file.

glasswall_runner.exe -m Export -i "C:\input\DOCXimages.docx" -o "C:\output\export\DOCXimages.docx.zip" -l "C:\azure\current\2023-06-05"

Export all files in a directory.

glasswall_runner.exe -m Export -i "C:\input" -o "C:\output\export_directory" -l "C:\azure\current\2023-06-05"

Import

Required args: m, i, o, l

Import a single file.

glasswall_runner.exe -m Import -i "C:\output\export\DOCXimages.docx.zip" -o "C:\output\import\DOCXimages.docx" -l "C:\azure\current\2023-06-05"

Import all files in a directory.

glasswall_runner.exe -m Import -i "C:\input" -o "C:\output\import_directory" -l "C:\azure\current\2023-06-05"

glasswall_runner.exe -m Import -i "C:\output\export_directory" -o "C:\output\import_directory" -l "C:\azure\current\2023-06-05"

Pack

Required args: m, i, o, l

Supported archive types include: "7z", "bz2", "gz", "tar", "xz", "zip".

Note: "rar" support is currently limited to unpacking only.

Pack a directory to ".zip".

glasswall_runner.exe -m Pack -i "C:\output\protect" -o "C:\output\pack" -l "C:\azure\current\2023-06-05"

The directory "C:\output\protect" is packed to "C:\output\pack\protect.zip"

Pack a directory to a different archive file format.

glasswall_runner.exe -m Pack -i "C:\output\protect" -o "C:\output\pack" -l "C:\azure\current\2023-06-05" --archive-file-type 7z

Unpack

Required args: m, i, o, l

Supported archive types include: "7z", "bz2", "gz", "rar", "tar", "xz", "zip".

Unpack a single archive.

glasswall_runner.exe -m Unpack -i "C:\output\pack\protect.zip" -o "C:\output\unpack" -l "C:\azure\current\2023-06-05"

The archive "C:\output\pack\protect.zip" is unpacked to "C:\output\unpack" maintaining the same directory structure. In this example the input archive "C:\output\pack\protect.zip" contained one file: "DOCXimages.docx" and is unpacked to the path: "C:\output\unpack\protect\DOCXimages.docx".

Unpack all archives in a directory.

glasswall_runner.exe -m Unpack -i "C:\output\pack" -o "C:\output\unpack_directory" -l "C:\azure\current\2023-06-05"

All archives in "C:\output\pack" are unpacked to "C:\output\unpack_directory" maintaining the same directory structure.

Note: In the above example the directory "C:\output\pack" contains two archives, "protect.7z" and "protect.zip". Each archive contains the same file: "DOCXimages.docx". Attempting to unpack both archives to the same output directory: "C:\output\unpack_directory" would result in one file overwriting the other if default settings are used. Files are processed alphabetically, "protect.7z" would be unpacked first resulting in the file path: "C:\output\unpack_directory\protect\DOCXimages.docx". "protect.zip" would be unpacked second and would overwrite the file at this path. This behaviour can be avoided using the "--unpack-include-file-type" command line argument. This will include the archive file type in the directory name when unpacking the archive and will result in two files present in the output directory: "C:\output\unpack_directory\protect.7z\DOCXimages.docx" and "C:\output\unpack_directory\protect.zip\DOCXimages.docx".

Combine

Required args: m, i, o

Combine all XML files in a directory into a single XML file.

glasswall_runner.exe -m Combine -i "C:\output\unpack_export\DOCXimages.docx" -o "C:\output\combine\DOCXimages.docx.xml"

Note: The input above is a directory containing XML files. It is the result of using Export mode followed by Unpack. See further example:
Transforming a file to XML.

Uncombine

Required args: m, i, o

Uncombine a single XML file to a directory.

glasswall_runner.exe -m Uncombine -i "C:\output\combine\DOCXimages.docx.xml" -o "C:\output\uncombine"

The directory "C:\output\uncombine\DOCXimages.docx" now contains all of the XML files that were combined in "C:\output\combine\DOCXimages.docx.xml".

Redact

Text content can be redacted from files using Redact mode with the --redact-list argument:

glasswall_runner.exe -m Redact -i "C:\input\redact\Test1234.pptx" -o "C:\output\redact\Test1234.pptx" -l "C:\azure\current\2023-06-05" --redact-list "C:\input\redact_list.txt"

please
lorem
microsoft

The default behaviour of Redact mode is to replace matched text content with asterisks: *. This can be customised by using a content management policy configuration file instead of the --redact-list argument:

glasswall_runner.exe -m Redact -i "C:\input\redact\Test1234.pptx" -o "C:\output\redact_custom\Test1234.pptx" -l "C:\azure\current\2023-06-05" -c "C:\input\config_redact.xml"

<?xml version="1.0" encoding="utf-8"?>
<config>
    <sysConfig>
        <interchange_type>xml</interchange_type>
    </sysConfig>
    <textSearchConfig libVersion="core2">
        <textList>
            <textItem>
                <text>please</text>
                <textSetting replacementChar="*">redact</textSetting>
            </textItem>
            <textItem>
                <text>lorem</text>
                <textSetting replacementChar="*">redact</textSetting>
            </textItem>
            <textItem>
                <text>microsoft</text>
                <textSetting replacementChar="*">redact</textSetting>
            </textItem>
        </textList>
    </textSearchConfig>
</config>

ExportUnpackCombine

Required args: m, i, o, l

Transform a file to XML

glasswall_runner.exe -m ExportUnpackCombine -i "C:\input\DOCXimages.docx" -o "C:\output\export_unpack_combine\DOCXimages.docx.xml" -l "C:\azure\current\2023-06-05"

The output XML file "C:\output\export_unpack_combine\DOCXimages.docx.xml" contains multiple file streams stored in CDATA.

If access to the intermediate files is required in between the use of Export, Unpack, Combine, see: Transforming a file to XML.

UncombinePackImportProtect

Required args: m, i, o, l

Transforming XML to a file

glasswall_runner.exe -m UncombinePackImportProtect -i "C:\output\export_unpack_combine\DOCXimages.docx.xml" -o "C:\output\uncombine_pack_import_protect\DOCXimages.docx" -l "C:\azure\current\2023-06-05"

If access to the intermediate files is required in between the use of Uncombine, Pack, Import, Protect, see: Transforming XML to a file.

UnpackProtectPack

Required args: m, i, o, l

Unpack an archive, Protect the contents, Pack to archive

glasswall_runner.exe -m UnpackProtectPack -i "C:\output\pack\protect.zip" -o "C:\output\unpack_protect_pack\protect.zip" -l "C:\azure\current\2023-06-05"

Files will be protected with the default sanitise all content management policy. A path to a custom content management policy configuration can be specified using the -c command line argument. Any files in the input archive that are nonconforming, unsupported, or that cannot be processed will be removed in the output archive. The --archive-whitelist argument can be used to specify a path to a .txt file containing a list of files that should never be removed.

.cfg
.gitattributes
.gitignore
.json
.log
.md
.sisl
.txt
.xml
.yaml
.yml

Specialised reporting is available for this mode using the --upp-report command line argument

glasswall_runner.exe -m UnpackProtectPack -i "C:\input\upp\nested2.zip" -o "C:\output\unpack_protect_pack2\nested2.zip" -l "C:\azure\current\2023-06-05" --archive-whitelist "C:\input\upp\archive_whitelist.txt" --upp-report "C:\output\unpack_protect_pack2\nested2.zip.upp.json"

{
    "summary": {
        "input_file": "C:\\input\\upp\\nested2.zip",
        "library": "archive_manager",
        "output_file": "C:\\output\\unpack_protect_pack2\\nested2.zip",
        "nested_archives": 3,
        "total_files": 7,
        "protected_files": 3,
        "whitelisted_files": 3,
        "quarantined_files": 1,
        "protected_archives": 1,
        "status": "Success"
    },
    "detail": [
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip",
            "library": "archive_manager",
            "status": "Success",
            "output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\tar.tar",
            "library": "archive_manager",
            "status": "Success",
            "output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip",
            "library": "archive_manager",
            "status": "Success",
            "output_directory": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.pptx",
            "library": "editor",
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.pptx"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.txt",
            "library": "editor",
            "whitelisted": true,
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.txt"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip\\c\\d.pptx",
            "library": "editor",
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.zip\\c\\d.pptx"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\a.zip\\b.zip\\c\\d.txt",
            "library": "editor",
            "whitelisted": true,
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\a.zip\\b.zip\\c\\d.txt"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\conforming\\launchCalculator.docx",
            "library": "editor",
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\conforming\\launchCalculator.docx"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\nonconforming\\NC-0000001.bmp",
            "library": "editor",
            "quarantined": true,
            "status": "FileRejected"
        },
        {
            "input_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\semumnh7\\nested2\\tar.tar\\tar.txt",
            "library": "editor",
            "whitelisted": true,
            "output_file": "C:\\Users\\Username\\AppData\\Local\\Temp\\glasswall\\irahr9yj\\nested2\\tar.tar\\tar.txt"
        }
    ]
}

The archive file format can be modified by changing the path specified in the -o command line argument. The example below would Unpack zip, Protect, and then Pack as 7z. For supported archive types see: Pack.

glasswall_runner.exe -m UnpackProtectPack -i "C:\output\pack\protect.zip" -o "C:\output\unpack_protect_pack\protect.7z" -l "C:\azure\current\2023-06-05"

CombinePerformanceTest

Required args: m, i, o

Combine multiple performance test JSON reports into a single report

glasswall_runner.exe -m CombinePerformanceTest -i "C:\output" -o "C:\output\combined_performance_test\output.pt.json"

By default this mode recursively searches the input directory for files ending .pt.json. This search string can be changed using the --combine-performance-test-filter command line argument:

glasswall_runner.exe -m CombinePerformanceTest -i "C:\output" -o "C:\output\combined_performance_test\output.pt.json" --combine-performance-test-filter ".json"

Further Examples

Transforming a file to XML

Transforming a file to XML using Export, Unpack, and Combine can be performed easily using the mode: ExportUnpackCombine. Alternatively if access to the intermediate files is required in between these modes, examples of using Export, Unpack, and Combine are below:

Starting with the file: "C:\input\DOCXimages.docx" we can first use Export mode to transform this file into multiple XML files within a zip archive:

glasswall_runner.exe -m Export -i "C:\input\DOCXimages.docx" -o "C:\output\export\DOCXimages.docx.zip" -l "C:\azure\current\2023-06-05"

The file has now been transformed to XML within a zip archive. If necessary, we can Unpack the zip archive:

glasswall_runner.exe -m Unpack -i "C:\output\export\DOCXimages.docx.zip" -o "C:\output\unpack_export" -l "C:\azure\current\2023-06-05"

The XML files are now present in the directory: "C:\output\unpack_export\DOCXimages.docx". If necessary, we can Combine these XML files into a single XML file:

glasswall_runner.exe -m Combine -i "C:\output\unpack_export\DOCXimages.docx" -o "C:\output\combine\DOCXimages.docx.xml"

The original file has now been transformed into a single XML file. The file size is expected to be larger than the original. For reference, the file used in this example was 6.56MB before transformation and 16.1MB after.

Transforming XML to a file

We can reverse the transformation using the mode: UncombinePackImportProtect. Alternatively if access to the intermediate files is required in between these modes, examples of using Uncombine, Pack, Import, Protect are below:

Starting with the XML file from the above example: "C:\output\combine\DOCXimages.docx.xml" we can first use Uncombine mode to transform this XML file into multiple XML files within a directory:

glasswall_runner.exe -m Uncombine -i "C:\output\combine\DOCXimages.docx.xml" -o "C:\output\uncombine"

The directory "C:\output\uncombine\DOCXimages.docx" now contains multiple XML files. We can pack them into a zip archive ready for Import to consume:

glasswall_runner.exe -m Pack -i "C:\output\uncombine\DOCXimages.docx" -o "C:\output\uncombine_packed" -l "C:\azure\current\2023-06-05"

The directory has been packed to "C:\output\uncombine_packed\DOCXimages.docx.zip". We can now use Import mode to transform the zip archive into a file.

glasswall_runner.exe -m Import -i "C:\output\uncombine_packed\DOCXimages.docx.zip" -o "C:\output\uncombine_imported\DOCXimages.docx" -l "C:\azure\current\2023-06-05"

The file has now been transformed from XML: "C:\output\uncombine_imported\DOCXimages.docx"