FSM: Aggregated Security Cache Hit Counters Reporting
- Former user (Deleted)
General
The aggregated security cache hit counters reporting introduces HERO stats (hero_stats_reports). HERO stats are a method of creating bulk client device activity reports.
HERO stats were originally designed as a mechanism to avoid sending immediate reports for repeated events. However, OpenSync still immediately reports the blocking events.
For example, a user has an ad blocker activated. The first time a user device accesses an advertising site (e.g., advertizeme.com ), a controller lookup indicates that the site should be blocked. FSM reports the decision right away and percolates it to the phone app so the user can decide to whitelist the site. All the subsequent access attempts to the site are blocked based on the Gatekeeper cache entry, and will still be reported right away.
FSM periodically gathers its Gatekeeper cache info and builds a batch report providing the access counter per cache entry or aggregated cache entries. In essence, FSM reports "This device accessed advertizeme.com N times over the last 15 minutes". This is what HERO stats are expected to provide.
Reported Content
Currently Provided Report Information
Without the HERO stats, the first attempt to access advertizeme.com is immediately reported as a tuple:
[
{
location id,
node id,
device mac,
event type (SNI/FQDN/URL/IP),
attribute value (sni/fqdn/url/IP),
action (blocked/allowed),
category as provided by gatekeeper,
timestamp
}
]Subsequent access attempts to advertizeme.com generate a similar immediate report.
HERO Report Proposal
With HERO stats, the first report follows the same model as shown above.
Further accesses are aggregated and reported in batches.
The aggregation can reduce the number of fields - or not, depending on the report requirements.
The most basic batch report is a collection of items as shown below:
[
location id,
node id,
{
device mac,
timestamp,
blocked events counter for the reporting window
},{
device mac,
timestamp,
allowed events counter for the reporting window
}
]The finest batch report is a collection of items as shown below:
[
location id,
node id,
{
device mac,event type (SNI/FQDN/URL/IP),
attribute value (sni/fqdn/url/IP),
action (blocked/allowed),
category as provided by gatekeeper,timestamp,
event counter for the reporting window
}
]Any aggregation in between is possible: per-event type, per-attribute value (i.e., advertizeme.com no matter if it comes from FQDN/URL/SNI). The current plan is to report every single cache entry.
Report Endpoint and Periodicity
Controller already configures the MQTT report endpoint and time interval.
Processing of Retry Attempts
There are various cases in which the attempt to reach a site might trigger more than one blocked event:
The browser/app may attempt to reach a site over both HTTPS and HTTP.
The browser/app might retry reaching the site on behalf of a user.
FSM could maintain a separate counter deduping the FQDN/SNI/Hostname accesses over a short period of time (a few dozens of seconds). This counter would represent the "perceived" access counter and would be reported alongside the accurate access counters maintained by the FW Gatekeeper cache.
However, as blocked events are still reported to the controller as soon as they happen, deduplicating of such events should be done at the controller level.