Open Caching Logging Specification

Project Status:

(roll over for info)

Start:

April 1, 2022

Estimated Completion:

May 31, 2024
  • Home
  • Open Caching Logging Specification

Problem Statement

This specification provides a framework for communication of CDN transactional logs between Open Caching participants to satisfy billing and operational requirements.

Project Description

CDNI Logging [RFC7937] defines requirements for exchange of log data between CDN participants including a Logging Reference Model, a Logging File Structure, and Logging Records along with an Atom-based index for retrieving the generated log files. Open Caching expands on the functionality defined in CDNI Logging to provide clear definitions and an expanded set of log record fields, additional mechanisms for transporting logs, new log container formats, log archive files, logging metadata sidecar files, new log record formats, configured inclusion of optional request and response header fields, and methods for advertising and configuring transformations of individual log fields. In the future, this specification will be expanded further to cover other OCS component logs (e.g. Request Router), custom log container formats and log record formats as well as uCDN controlled configuration for aggregation, filtering, and sampling of log records.

Project Type

Document

Project Leads

Advisors

There are no SMEs associated with this project.

Draft Documents

Estimated Publication Date: Q2 2023

(DRAFT) SVTA2050: Open Caching Logging Specification

This specification provides a framework for communication of CDN transactional logs between Open Caching participants to satisfy billing and operational requirements.


Goals and Objectives

  • Provide a framework for communication of CDN request logs between Open Caching participants to satisfy billing and operational requirements.
  • Clarify and extend missing functionality from the CDNI Logging Interface defined in RFC 7937.
  • Allow batch push and streaming modes of operation in addition to the pull mode described in CDNI.
  • Provide useful default and minimal configurations for the most common use cases, but allow custom configuration when the pre-defined configurations are not suitable.

Project Scope

This specification covers the following:
  • Extensions to the pull mode of operation described in RFC7937
  • Log container/file formats
    • ELF
    • JSON
    • Newline delimited records (\n, \n\r)
    • Archive containers (tar) containing other log files
  • Log record fields
  • Standard log record types
    • A standard log record type is a combination of a set of fields in a defined textual format (JSON, CSV, or whitespace delimited).
    • A ‘minimal’ type optimized for data size, including only the fields necessary to perform billing functions.
    • A ‘standard’ type with a set of fields likely to be useful for operational monitoring.
    • An ‘extended’ type with an exhaustive field set likely to cover most operational and troubleshooting use cases.
  • Field transformations to support obfuscation, truncation, hashing, and encryption operations
  • Optional custom request and response header field inclusion
  • Logging transports supporting both push and pull modes of operation
    • S3
    • SFTP
    • Kafka
    • Generic custom transport
This specification does NOT cover the following, but these items may be addressed in a future draft:
  • Aggregation
  • Filtering
  • Auth
  • Custom container formats
  • Custom log record formats
  • Custom field definitions

Contributors

The following members have contributed to this project. Click on their name to visit their profile. If they have not published their profile, the link will redirect to their LinkedIn profile.

Presentations

The following presentations delivered during Open Caching working group sessions may provide additional information about this project.

Have A Question ABout Membership?

Schedule A Meeting

Send An Email

Don’t want to schedule a face-to-face meeting just now? No problem. Simply send your membership question to info@streamingvideoalliance.org or fill out the form below and someone will get back to you as soon as possible.

"*" indicates required fields

Name*
Email*
Hidden