compress()

Compress an array of log lines into semantic templates — the main entry point for most callers.

Compress an array of log lines into semantic templates.

Signature

function compress(
  lines: string[],
  options?: CompressOptions
): CompressionResult

Parameters

`lines`

Type: string[]
Required: Yes

Array of log lines to compress. Each line should be a complete log entry. Blank and whitespace-only lines are skipped and are not counted in stats.inputLines.

`options`

Type: CompressOptions
Required: No

interface CompressOptions {
  format?: OutputFormat;
  maxTemplates?: number;
  drain?: DrainOptions;
}

Only three keys are accepted at the top level. Every Drain algorithm parameter lives inside drain.

`options.format`

Type: 'summary' | 'detailed' | 'json' | 'json-stable'
Default: 'summary'

Output format for the formatted field in the result. result.templates is populated identically whatever the format:

'summary' - Compact template list with frequencies and a rare-events section
'detailed' - Full templates with sample variables and all diagnostic metadata
'json' - Machine-readable JSON with a version field, pretty printed
'json-stable' - Same data with recursively sorted keys and no whitespace, for stable diffs and LLM prompt-cache hits

Maximum number of templates to include in both result.templates and the formatted output. Templates are sorted by occurrence count (most frequent first) before truncation. stats.uniqueTemplates still reports the untruncated count.

`options.drain`

Type: DrainOptions
Default: {}

Drain algorithm configuration:

const result = compress(logs, {
  drain: {
    depth: 5,
    simThreshold: 0.5,
  },
});

Option	Type	Default	Meaning
`depth`	`number`	`4`	Parse tree depth. Higher values create more specific templates.
`simThreshold`	`number`	`0.4`	Similarity required to join an existing template, `0.0`–`1.0`. Lower groups more aggressively. When omitted, the parsing strategy decides and may vary it by depth.
`maxChildren`	`number`	`100`	Max children per parse tree node. At capacity, further tokens collapse into a wildcard branch.
`maxClusters`	`number`	`1000`	Max total templates. Once reached, unmatched lines are discarded and counted in `stats.droppedLines`.
`maxSamples`	`number`	`3`	Max sample variables stored per template.
`preprocessing`	`ParsingStrategy`	built-in	Custom preprocessing strategy. See Custom Preprocessing.
`onProgress`	`ProgressCallback`	`undefined`	Progress callback, see below.

`options.drain.onProgress`

type ProgressCallback = (event: ProgressEvent) => void;

interface ProgressEvent {
  processedLines: number;
  totalLines?: number;
  currentPhase: 'parsing' | 'clustering' | 'finalizing';
  percentComplete?: number;
}

At most ~100 events are emitted for a given call.

Return Value

Returns a CompressionResult object:

interface CompressionResult {
  templates: Template[];
  stats: {
    inputLines: number;
    uniqueTemplates: number;
    compressionRatio: number;
    estimatedTokenReduction: number;
    droppedLines?: number;
    processingTimeMs?: number;
  };
  formatted: string;
}

`templates`

Array of extracted templates, sorted by occurrence count (descending) and truncated to maxTemplates.

See Template interface for details.

`stats`

inputLines - Non-blank input log lines processed
uniqueTemplates - Number of unique templates discovered (before maxTemplates truncation)
compressionRatio - 1 - (uniqueTemplates / inputLines), clamped to 0.0–1.0. Higher means more compression.
estimatedTokenReduction - Estimated reduction as a ratio between 0 and 1, not a percentage. Derived from a character-count proxy: each pattern's length times its occurrence count, versus the pattern printed once.
droppedLines - Lines discarded because maxClusters was reached. Non-zero means the output is incomplete and compressionRatio overstates the real result.
processingTimeMs - Wall-clock processing time. Populated by compress() and compressText(); not populated by Drain.getResult().

import { compress } from 'logpare';

const logs = [
  'ERROR Connection to 192.168.1.100 failed',
  'ERROR Connection to 192.168.1.101 failed',
  'INFO Request 10001 completed',
  'INFO Request 10002 completed',
];

const result = compress(logs);

console.log(result.formatted);
// === Log Compression Summary ===
// Input: 4 lines → 2 templates (50.0% reduction)
//
// Top templates by frequency:
// 1. [2x] ERROR Connection to <*> failed
// 2. [2x] INFO Request <*> completed

With Options

const result = compress(logs, {
  format: 'detailed',
  maxTemplates: 100,
  drain: {
    depth: 5,
    simThreshold: 0.5,
  },
});

Progress Tracking

const result = compress(logs, {
  drain: {
    onProgress: (event) => {
      console.log(`Phase: ${event.currentPhase}`);
      console.log(`Processed: ${event.processedLines} lines`);
      if (event.percentComplete !== undefined) {
        console.log(`Progress: ${event.percentComplete.toFixed(1)}%`);
      }
    },
  },
});

Accessing Templates

const result = compress(logs);

// Filter error templates
const errors = result.templates.filter(t => t.severity === 'error');

// Get most frequent template
const mostFrequent = result.templates[0];
console.log(`Most common: ${mostFrequent.pattern} (${mostFrequent.occurrences}x)`);

// Extract all URLs
const allUrls = result.templates.flatMap(t => t.urlSamples);

Reading from File

import { readFileSync } from 'node:fs';
import { compress } from 'logpare';

const logContent = readFileSync('app.log', 'utf-8');
const lines = logContent.split(/\r?\n/);

const result = compress(lines, {
  format: 'detailed',
  maxTemplates: 20,
});

console.log(result.formatted);

Checking for Truncation

const result = compress(logs, { drain: { maxClusters: 100 } });

if ((result.stats.droppedLines ?? 0) > 0) {
  console.warn(
    `${result.stats.droppedLines} lines dropped — raise maxClusters for full coverage`
  );
}