MANRS Community

Download

manrs-community@elists.manrs.org

January 2024

2 participants
2 discussions

Re: [manrs-community] Upcoming changes to the MANRS measurement framework
by Pablo Lopez-Aguilar Beltran Jan. 19, 2024

Jan. 19, 2024

Dear MANRS participants, We are glad to inform you that the features we announced back in December are now deployed in the MANRS Observatory. We also updated the Measurement framework https://www.manrs.org/manrs-observatory/measurement-framework/ to reflect these changes. Again, we encourage you to use the *feedback function* in the detailed reports to remove possible false positives and improve your MANRS readiness score. Best regards, • • • • *Pablo López-Aguilar* *Operations and Business Associate Director, Internet Integrity Program* Global Cyber Alliance plopezaguilar(a)globalcyberalliance.org +34 670527760 [image: GCA Website] <https://www.globalcyberalliance.org/> [image: GCA Twitter] <https://twitter.com/GlobalCyberAlln> [image: GCA Facebook] <https://www.facebook.com/GlobalCyberAlliance/> [image: GCA LinkedIn] <https://www.linkedin.com/company/global-cyber-alliance/> [image: GCA YouTube] <https://www.youtube.com/channel/UCC5x4cUnZqWsV7OqBx1vCOQ> [image: GCA GitHub] <https://github.com/GlobalCyberAlliance/> [image: GCA Email Signup] <https://www.globalcyberalliance.org/#signup> [image: GCA Instagram] <https://www.instagram.com/globalcyberalliance/> [image: GCA Forums] <https://community.globalcyberalliance.org/> GCA takes your privacy seriously. To review our privacy policy, please click here <https://www.globalcyberalliance.org/privacy-policy/>. On 14/12/2023, 15:03, "Andrei Robachevsky" <robachevsky(a)isoc.org> wrote: Dear MANRS participants, In July 2023, we started publishing the MANRS Readiness Scores <https://www.manrs.org/2023/06/manrs-readiness-scores-to-be-made-public-for-…> for Actions 2, 3, and 4 (IRR and RPKI) for participants in the NetOps program. We didn't publish Action 1 scores at the time due to legitimate concerns within the community that the data used to calculate the scores produced too many false positives. As I presented during the MANRS Community Meeting on 28 November, we've done several improvements in the quality of these data and the scores. Below, you will find a summary of the upcoming changes — more detailed information can be found here: https://www.manrs.org/2023/12/the-road-to-publishing-action-1-netops-scores/. Please review these changes and let us know your questions or concerns. We plan to deploy these changes in January 2024. Note: Action 1 scores *will not be published yet* on the MANRS website to allow participants ample time to assess the changes' actual impact on the readiness scores. 1. The M1 and M1C metrics will no longer be included in the Action 1 score calculation. 2. The M2 and M2C metrics derived from BGPStream data have been removed completely from the Observatory due to low confidence. 3. The M2 and M2C metrics derived from GRIP data will be filtered based on the RPKI and IRR data to remove many false positives. 4. The M3 and M3C metrics will also be filtered. We will split all bogon prefixes into "full" and "administrative" bogons. The reason for doing so is that the operational practice is that "administrative bogons" are still being routed in some cases. Only "full" bogons will be considered when calculating the scores. 5. The M4 and M4C metrics will be filtered into "full" and "administrative bogons in the same way. These changes will lead to higher confidence in the data and the scores. Statistical analysis of filtered data revealed no false positives and, in some cases, led to an 80% reduction in reported incidents. We understand that false positives cannot be eliminated completely. To give you an opportunity to flag residual false positives a “feedback” function <https://www.manrs.org/2021/07/new-data-source-feedback-loop-enhance-manrs-o…> has been in place since July 2021. After being reviewed by an auditor, such incidents will be removed. We encourage you to use *this function when reviewing your report*. We always welcome feedback on the MANRS Observatory and any of its data, whether due to these changes or for any other reason. You can send your feedback to this mailing list, email contact(a)manrs.org, or submit feedback via the website <https://www.manrs.org/about/contact/>. Regards, Andrei Robachevsky MANRS Secretariat

1 0

Observatory December data outage
by Andy Clyde Jan. 11, 2024

Jan. 11, 2024

Hi all, We had an outage with the CIDR bogon report towards the end of December. This meant that we were unable to update the December data for several days. Partial December data was available until 4th January but it was then removed to avoid scores based on the partial data being published. We worked with CIDR to re-download the correct data for the period of the outage and have reprocessed all December and January data in the Observatory. It is all now available and the full December scores have been published. Thanks Andy Clyde MANRS Senior Application Developer

1 0