Add Minified File Detection Check (Internal.Tokenizer.Exception)

[davidperezgar]

Overview

Add a check to detect minified PHP files that cannot be properly processed by the PHP tokenizer. When minified PHP files are detected, the check should report an error indicating that the non-minified source file must be included in the plugin.

What This Check Does

The Internal.Tokenizer.Exception is a built-in PHP_CodeSniffer rule that triggers when the PHP tokenizer encounters an unrecoverable error while parsing a file. This most commonly occurs with:

1. Minified PHP files - PHP code with all whitespace, comments, and line breaks removed
2. Severely malformed code - Syntax that prevents proper tokenization
3. Encoding issues - Files with incorrect or mixed character encodings

For WordPress plugins, minified PHP files are problematic because:

• They cannot be properly reviewed for security issues
• They obscure malicious code
• They prevent proper code analysis
• Plugin reviews require readable, reviewable code

WordPress.org Plugin Guidelines

According to the WordPress Plugin Handbook:

Minified scripts/files are permitted, but the non-minified versions must also be included in the theme/plugin.

Example of Problematic Code

Minified PHP File (NOT ALLOWED without original)

<?php
function test(){global $wpdb;$foo=$_POST['id'];$wpdb->query("SELECT * FROM $wpdb->posts WHERE ID='$foo'");return true;}add_action('init','test');

This single-line code is virtually unreadable and cannot be properly analyzed for security issues.

Proper PHP File (REQUIRED)

<?php
/**
 * Test function
 */
function test() {
    global $wpdb;
    
    $foo = $_POST['id'];
    $wpdb->query( "SELECT * FROM $wpdb->posts WHERE ID='$foo'" );
    
    return true;
}

add_action( 'init', 'test' );

Implementation Approach

Unlike custom sniffs, Internal.Tokenizer.Exception is a built-in PHP_CodeSniffer error code that automatically triggers when the tokenizer fails. We just need to:

1. Include the rule in the plugin-check ruleset
2. Customize the error message to be WordPress.org-specific
3. Create test fixtures to ensure the check works correctly

Ruleset Configuration

Add to phpcs-sniffs/PluginCheck/ruleset.xml:

<!-- Minified PHP files are not allowed without the original source -->
<rule ref="Internal.Tokenizer.Exception">
    <message>File appears to be minified or has tokenization errors and cannot be processed. If this is a minified file, the non-minified source file must be included in the plugin.</message>
    <type>error</type>
</rule>