Open Caching Logging Specification

Open Caching

Project Status:

(roll over for info)

Completed

The project has been completed.

Start:

April 1, 2022

Estimated Completion:

June 3, 2024
  • Home
  • Open Caching Logging Specification

Problem Statement

This specification provides a framework for communication of CDN transactional logs between Open Caching participants to satisfy billing and operational requirements.

Project Description

CDNI Logging [RFC7937] defines requirements for exchange of log data between CDN participants including a Logging Reference Model, a Logging File Structure, and Logging Records along with an Atom-based index for retrieving the generated log files. Open Caching expands on the functionality defined in CDNI Logging to provide clear definitions and an expanded set of log record fields, additional mechanisms for transporting logs, new log container formats, log archive files, logging metadata sidecar files, new log record formats, configured inclusion of optional request and response header fields, and methods for advertising and configuring transformations of individual log fields. In the future, this specification will be expanded further to cover other OCS component logs (e.g. Request Router), custom log container formats and log record formats as well as uCDN controlled configuration for aggregation, filtering, and sampling of log records.

Project Type

Document

Project Leads

Ben Rosenblum

Advisors

There are no SMEs associated with this project.

Published Documents

Version: 1.0

Date Plublished: 06/03/2024

More Details

Add to Basket

SVTA2050: Open Caching Logging Specification

This document describes extensions to the Open Caching Configuration API and the Content Delivery Networks Interconnection (CDNI) Footprint & Capabilities Interface [RFC8008] to support configuration and advertisement of logging capabilities, along with examples and best practices to support logging,log record transformation and transmission between participants within an Open Caching environment. The extensions are based on the Internet Engineering Task Force (IETF) CDNI Request for Comments (RFCs) and extensions proposed by this Streaming Video Technology Alliance (SVTA) group.

Goals and Objectives

  • Provide a framework for communication of CDN request logs between Open Caching participants to satisfy billing and operational requirements.
  • Clarify and extend missing functionality from the CDNI Logging Interface defined in RFC 7937.
  • Allow batch push and streaming modes of operation in addition to the pull mode described in CDNI.
  • Provide useful default and minimal configurations for the most common use cases, but allow custom configuration when the pre-defined configurations are not suitable.

Project Scope

This specification covers the following:
  • Extensions to the pull mode of operation described in RFC7937
  • Log container/file formats
    • ELF
    • JSON
    • Newline delimited records (\n, \n\r)
    • Archive containers (tar) containing other log files
  • Log record fields
  • Standard log record types
    • A standard log record type is a combination of a set of fields in a defined textual format (JSON, CSV, or whitespace delimited).
    • A ‘minimal’ type optimized for data size, including only the fields necessary to perform billing functions.
    • A ‘standard’ type with a set of fields likely to be useful for operational monitoring.
    • An ‘extended’ type with an exhaustive field set likely to cover most operational and troubleshooting use cases.
  • Field transformations to support obfuscation, truncation, hashing, and encryption operations
  • Optional custom request and response header field inclusion
  • Logging transports supporting both push and pull modes of operation
    • S3
    • SFTP
    • Kafka
    • Generic custom transport
This specification does NOT cover the following, but these items may be addressed in a future draft:
  • Aggregation
  • Filtering
  • Auth
  • Custom container formats
  • Custom log record formats
  • Custom field definitions

Contributors

The following members have contributed to this project. Click on their name to visit their profile. If they have not published their profile, the link will redirect to their LinkedIn profile.

Alfonso Siloniz

Ben Rosenblum

Dave Seddon

Kent Seward

Matt Stock

Omar Ramadan

Rajeev RK

Additional References

Presentations

The following presentations delivered during Open Caching working group sessions may provide additional information about this project.

Get Notified!

Want to get notified when we publish a new blog post, technical document, webinar, or other piece of content? Just enter your email address below. Don’t worry, we won’t share your email with anyone and will only use it to notify you of new content.

Resources

Legal

Other Sites

Making Streaming Video Better

The Streaming Video Technology Alliance is committed to bringing video streaming companies together to help build a better viewer experience at scale.

© The Streaming Video Technology Alliance (SVTA).

SVTA sites are hosted on the Qwilt Open Caching CDN.

X-twitter Vimeo Linkedin
Close

Have A Question ABout Membership?

Schedule A Meeting

Send An Email

Don’t want to schedule a face-to-face meeting just now? No problem. Simply send your membership question to info@streamingvideoalliance.org or fill out the form below and someone will get back to you as soon as possible.

"*" indicates required fields

Name*
Email*
Hidden