Skip to main content

Documents Module - API Specification

Version: v1.1.9 | Status: Active | Last Updated: February 2026

Overview​

This document provides detailed API specification for the Documents module, including all public functions, classes, and data structures.

Core Document Operations​

read_document​

def read_document(
    file_path: str | Path,
    format: Optional[DocumentFormat] = None,
    encoding: Optional[str] = None,
) -> Document

Read a document from a file with automatic format and encoding detection.

Parameters:

  • file_path (str | Path): Path to the document file
  • format (Optional[DocumentFormat]): Optional format hint. If None, format is auto-detected from file extension
  • encoding (Optional[str]): Optional encoding hint. If None, encoding is auto-detected

Returns: Document object with content and metadata

Raises:

  • DocumentReadError: If reading fails
  • UnsupportedFormatError: If format is not supported

Example:

from codomyrmex.documents import read_document

doc = read_document("example.md")
print(doc.content)

write_document​

def write_document(
    document: Document,
    file_path: str | Path,
    format: Optional[DocumentFormat] = None,
    encoding: Optional[str] = None,
) -> None

Write a document to a file.

Parameters:

  • document (Document): Document object to write
  • file_path (str | Path): Path where document should be written
  • format (Optional[DocumentFormat]): Optional format override. Uses document.format if not provided
  • encoding (Optional[str]): Optional encoding override. Uses document.encoding if not provided

Raises:

  • DocumentWriteError: If writing fails
  • UnsupportedFormatError: If format is not supported

Example:

from codomyrmex.documents import write_document, Document, DocumentFormat

doc = Document(content="Hello, World!", format=DocumentFormat.MARKDOWN)
write_document(doc, "output.md")

parse_document​

def parse_document(
    content: str,
    format: DocumentFormat,
    file_path: str | None = None,
) -> Document

Parse content string into a Document object.

Parameters:

  • content (str): Content string to parse
  • format (DocumentFormat): Format of the content
  • file_path (str | None): Optional file path for context

Returns: Document object

Raises:

  • DocumentParseError: If parsing fails

validate_document​

def validate_document(
    document: Document,
    schema: Optional[dict] = None,
) -> ValidationResult

Validate a document against a schema or format rules.

Parameters:

  • document (Document): Document to validate
  • schema (Optional[dict]): Optional JSON schema for validation

Returns: ValidationResult with validation status and any errors/warnings

Format-Specific Functions​

Markdown​

def read_markdown(file_path: str | Path, encoding: Optional[str] = None) -> str
def write_markdown(content: str, file_path: str | Path, encoding: Optional[str] = None) -> None

JSON​

def read_json(
    file_path: str | Path,
    encoding: Optional[str] = None,
    schema: Optional[dict] = None,
) -> dict

def write_json(
    data: dict,
    file_path: str | Path,
    encoding: Optional[str] = None,
    indent: int = 2,
    ensure_ascii: bool = False,
) -> None

YAML​

def read_yaml(file_path: str | Path, encoding: Optional[str] = None) -> dict
def write_yaml(
    data: dict,
    file_path: str | Path,
    encoding: Optional[str] = None,
    default_flow_style: bool = False,
) -> None

PDF​

def read_pdf(file_path: str | Path) -> PDFDocument
def write_pdf(
    content: str,
    file_path: str | Path,
    metadata: Optional[dict] = None,
) -> None

Text​

def read_text(file_path: str | Path, encoding: Optional[str] = None) -> str
def write_text(content: str, file_path: str | Path, encoding: Optional[str] = None) -> None

HTML​

def read_html(file_path: str | Path, encoding: Optional[str] = None) -> str
def write_html(content: str, file_path: str | Path, encoding: Optional[str] = None) -> None
def strip_html_tags(html_content: str) -> str

Read/write HTML files and strip HTML tags to extract plain text.

XML​

def read_xml(file_path: str | Path, encoding: Optional[str] = None) -> str
def write_xml(content: str, file_path: str | Path, encoding: Optional[str] = None) -> None

Read/write XML files. read_xml validates the content parses as XML.

CSV​

def read_csv(file_path: str | Path, encoding: Optional[str] = None) -> list[dict]
def write_csv(
    data: list[dict],
    file_path: str | Path,
    encoding: Optional[str] = None,
    fieldnames: Optional[list[str]] = None,
) -> None

Read CSV as a list of dictionaries (keyed by header names). Write CSV from a list of dictionaries.

Transformation Functions​

convert_document​

def convert_document(document: Document, target_format: DocumentFormat) -> Document

Convert a document to a different format.

Parameters:

  • document (Document): Document to convert
  • target_format (DocumentFormat): Target format

Returns: New Document in target format

Raises:

  • DocumentConversionError: If conversion fails
  • UnsupportedFormatError: If conversion is not supported

merge_documents​

def merge_documents(
    documents: List[Document],
    target_format: DocumentFormat = None,
) -> Document

Merge multiple documents into a single document.

Parameters:

  • documents (List[Document]): List of documents to merge
  • target_format (DocumentFormat): Optional target format. Uses first document's format if not provided

Returns: Merged Document

Raises:

  • DocumentConversionError: If merging fails
  • ValueError: If documents list is empty

split_document​

def split_document(document: Document, criteria: dict) -> List[Document]

Split a document into multiple documents based on criteria.

Parameters:

  • document (Document): Document to split
  • criteria (dict): Split criteria. Options:
    • {"method": "by_sections"} - Split by markdown sections
    • {"method": "by_size", "max_size": 10000} - Split by character size
    • {"method": "by_lines", "lines_per_chunk": 100} - Split by number of lines

Returns: List of split Document objects

format_document​

def format_document(document: Document, style: str = "default") -> Document

Format a document according to a style.

Parameters:

  • document (Document): Document to format
  • style (str): Formatting style - "default", "compact", or "pretty"

Returns: Formatted Document

Search Functions​

InMemoryIndex​

class InMemoryIndex:
    def add(self, document: Document) -> None
    def remove(self, doc_id: str) -> None
    def search(self, terms: list[str]) -> list[str]
    def get_document(self, doc_id: str) -> Optional[Document]
    def save(self, path: Path) -> None
    @classmethod
    def load(cls, path: Path) -> "InMemoryIndex"
    @property
    def document_count(self) -> int

In-memory inverted index for document search. Supports add, remove, term-based search (AND semantics), and JSON serialization.

index_document​

def index_document(document: Document, index: Optional[InMemoryIndex] = None) -> InMemoryIndex

Index a document for search. Creates a new index if none provided.

create_index​

def create_index() -> InMemoryIndex

Create a new empty search index.

search_documents​

def search_documents(query: str, index: InMemoryIndex) -> List[Document]

Search documents using a query string. Returns matching Document objects.

search_index​

def search_index(query: str, index: InMemoryIndex) -> List[dict]

Search index and return results with TF-based scores. Returns list of dicts with document_id, score, and document keys, sorted by score descending.

QueryBuilder​

class QueryBuilder:
    def add_term(self, term: str) -> "QueryBuilder"
    def add_filter(self, field: str, value: str) -> "QueryBuilder"
    def set_sort(self, field: str) -> "QueryBuilder"
    def build(self) -> str
    def to_dict(self) -> dict

Fluent builder for constructing search queries.

build_query​

def build_query(terms: List[str], filters: dict = None, sort_by: str = None) -> str

Convenience function for building a query string from terms.

Metadata Functions​

extract_metadata​

def extract_metadata(file_path: str | Path) -> dict

Extract metadata from a document file (file system metadata + format-specific).

update_metadata​

def update_metadata(file_path: str | Path, metadata: dict) -> None

Update metadata for a document file (supports markdown frontmatter).

get_document_version / set_document_version​

def get_document_version(file_path: str | Path) -> Optional[str]
def set_document_version(file_path: str | Path, version: str) -> None

Data Models​

Document​

@dataclass
class Document:
    content: Any              # str, dict, list, bytes depending on format
    format: DocumentFormat
    file_path: Optional[Any] = None
    encoding: Optional[str] = None
    metadata: Any = None
    id: str = field(default_factory=lambda: uuid.uuid4().hex)
    document_type: Optional[DocumentType] = None  # auto-derived from format
    created_at: Optional[datetime] = field(default_factory=datetime.now)
    modified_at: Optional[datetime] = field(default_factory=datetime.now)

    @property
    def type(self) -> DocumentType          # shorthand for document_type
    def get_content_as_string(self) -> str  # serialize content to string
    def to_dict(self) -> dict               # serialize document to dict

DocumentFormat​

class DocumentFormat(Enum):
    MARKDOWN = "markdown"
    TEXT = "text"
    HTML = "html"
    JSON = "json"
    XML = "xml"
    YAML = "yaml"
    CSV = "csv"
    PDF = "pdf"
    RTF = "rtf"
    DOCX = "docx"
    XLSX = "xlsx"
    PY = "py"
    JS = "js"

DocumentType​

class DocumentType(Enum):
    TEXT = "text"
    MARKUP = "markup"
    STRUCTURED = "structured"
    BINARY = "binary"
    CODE = "code"

DocumentMetadata​

@dataclass
class DocumentMetadata:
    title: Optional[str] = None
    author: Optional[str] = None
    created_at: Optional[datetime] = None
    modified_at: Optional[datetime] = None
    version: Optional[str] = None
    tags: list[str] = field(default_factory=list)
    custom_fields: dict[str, Any] = field(default_factory=dict)

    def copy(self) -> "DocumentMetadata"
    def to_dict(self) -> dict
    @classmethod
    def from_dict(cls, data: dict) -> "DocumentMetadata"

MetadataField​

@dataclass
class MetadataField:
    name: str
    value: Any
    data_type: Optional[str] = None
    source: Optional[str] = None

ValidationResult​

class ValidationResult:
    is_valid: bool
    errors: list[str]
    warnings: list[str]

Supports truthiness: if result: checks is_valid.

Configuration​

DocumentsConfig​

class DocumentsConfig:
    default_encoding: str = "utf-8"
    max_file_size: int = 100 * 1024 * 1024  # 100MB
    enable_caching: bool = True
    cache_directory: Optional[Path] = None
    strict_validation: bool = False

Cache directory defaults to ~/.codomyrmex/documents_cache or uses CODOMYRMEX_CACHE_DIR environment variable.

get_config / set_config​

def get_config() -> DocumentsConfig
def set_config(config: DocumentsConfig) -> None