dataRisk-detection-resources
English | 简体中文
With the release of China's "Data Security Protection Law" in 2021, it means that data security is expected to form a new outlet in China.
The author is fortunate to join one of China's leading data security startups in 2021, engaged in cutting-edge research and implementation of data science combined with data security. In the process of exploration, I found that there are not many materials on the Internet specifically for data security, so I came up with the idea of arranging relevant materials and thinking, hoping to do my best to promote the development of the community.
Refuse to prostitute, welcome star!！
A person can go fast, only a group of people can go farther. The author has set up a big data security technology exchange group, with friends all over Silicon Valley, Singapore, Tencent, Ali, Zhejiang University, etc. Like-minded friends are welcome to contact me to join!
Last updated date is:2022/11
AI Application Defense
Using AI for Application Security Protection
personal collection：Data security intelligent risk control landing practice
- isc2022
- https://mp.weixin.qq.com/s/Ce8iXvAuNf2n3OFZSmFi1Q
- https://zhuanlan.zhihu.com/p/466955597?
- https://zhuanlan.zhihu.com/p/511095084
- https://mp.weixin.qq.com/s/Sme4gLnEHyxyhRSN2RUqCA
- https://www.zhihu.com/column/c_1471819989803700224
Getting Started Overview
- Why Machine Learning Always Fails to Solve Cybersecurity Problems: Talk About Feature Space
- Why Machine Learning Always Fails to Solve Cybersecurity Problems: Fragile Systems Engineering
- Why Machine Learning Always Fails to Solve Cybersecurity Problems: Unreasonable Evaluation Metrics
- Why Machine Learning Always Fails to Solve Cybersecurity Problems: Machine Learning Is Not a Panacea
- Explainable Machine Learning for Solving Cybersecurity Problems
OWASP10
API Risk Discovery System
- https://mp.weixin.qq.com/s/-9xkAROp7_A6gDjTLxfUsg
- https://www.freebuf.com/articles/web/189981.html
- https://search.freebuf.com/search/?search=%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0#article
- Exploration of sensitive information leakage governance based on machine learning
Risk business
- https://mp.weixin.qq.com/s/xGY1PxoH9Tlio2mWH7QLjw
- Du Yuejin: Basic Ideas of Data Security Governance
- Data Security Composite Governance and Practice White Paper
- Parameter tampering and traffic replay
- Common API Attacks
Malicious registered account
- "Unveiling Fake Accounts at the Time of Registration: An Unsupervised Approach"
- "DeepScan: Exploiting Deep Learning for Malicious Account Detection in Location-Based Social Networks"
Malicious Mail
Malicious traffic detection
- Machine Learning for Malicious Traffic Detection Feature Engineering
- Machine Learning KNN Detects Malicious Traffic
Machine Learning and Security
- Webshell detection method combining reinforcement learning and CNN
- Security Risks of Automated Machine Learning
- Using open source intelligence to detect and explain malicious behavior
- BadNL: Semantic Preserving Improved NLP Model Backdoor Attack
- APTMalInsight: Identifying and Recognizing APT Malware Based on System Call Information and Ontology Knowledge Framework
Graph Data Mining
- 《A Practical Approach to Constructing a Knowledge Graph for Cybersecurity》
- 《Developing an Ontology for Cyber Security Knowledge Graphs》
- 《Towards a Relation Extraction Framework for Cyber-Security Concepts》
- https://zhuanlan.zhihu.com/p/406415230
- https://zhuanlan.zhihu.com/p/69159780
- https://zhuanlan.zhihu.com/p/74274673
- https://zhuanlan.zhihu.com/p/75123819
marchine learning for UEBA
- 《AI2: Training a big data machine to defend》
- 《Big Data Security Challenges: An Overview and Application of User Behavior Analytics》
- 《Adaptive Intrusion Detection System via Online Learning》
- 《A multi-model approach to the detection of web-based attacks》
- 《McPAD : A Multiple Classifier System for Accurate Payload-based Anomaly Detection》
- 《Using Generalization and Characterization Techniques in the Anomaly-based Detection of Web Attacks》
- 《Anomaly-Based Web Attack Detection: A Deep Learning Approach》
- 《A Big Data Analysis Framework for Model-Based Web User Behavior Analytics》
- 《Anomalous Payload-based Network Intrusion Detection》
- 《Data mining for security at Google》
- 《User and Entity Behavior Analytics for Enterprise Security》
- 《A Comprehensive Approach to Intrusion Detection Alert Correlation》
- 《Trafc Anomaly Detection Using K-Means Clustering》
- 《Calculation of the Behavior Utility of a Network System: Conception and Principle》
- 《Spectrogram: A Mixture-of-Markov-Chains Model for Anomaly Detection in Web Traffic》
- 《用户画像相关技术》
MLOPS
Intrusion Detection
- Web attack classification and detection model based on machine learning
- https://blog.cloudflare.com/api-abuse-detection/
- Using Machine Learning to Detect Malicious HTTP Outbound Traffic
- ExecScent: Mining for New C&C Domains in Live Networks with Adaptive Control Protocol Templates
- MADE: Security Analytics for Enterprise Threat Detection
- Machine Learning Practices in Internet Giants
- Application of Machine Learning in Intrusion Detection - Training Intrusion Detection Discriminant Model Based on ADFA-LD Training Set
- datacon competition direction three - attack source and attacker analysis writeup
- [Machine learning-based malware encryption traffic detection research sharing](https://blog.riskivy.com/%e5%9f%ba%e4%ba%8e%e6%9c%ba%e5%99%a8% e5%ad%a6%e4%b9%a0%e7%9a%84%e6%81%b6%e6%84%8f%e8%bd%af%e4%bb%b6%e5%8a%a0%e5% af%86%e6%b5%81%e9%87%8f%e6%a3%80%e6%b5%8b/?from=groupmessage&isappinstalled=0)
- anomaly-detection-through-reinforcement-learning
Malicious url detection
- URLNet: Learning URL Representations via Deep Learning for Malicious URL Detection
- My AI Security Detection Study Notes (1)
- "Compromised or Attacker-Owned: A Large Scale Classification and Study of Hosting Domains of Malicious URLs"
DDOS
- Predicting DDoS attacks based on KDDCUP 99 dataset
- Research on DDoS Attack Detection Technology Based on Spectral Analysis and Statistical Machine Learning
- Research on Distributed Denial of Service Attack Detection Method Based on Machine Learning
- DDoS Attacks Using Hidden Markov Models and Cooperative ReinforcementLearning*
Botnet Detection
- [Win the 0-Day Racing Game Against Botnet on Cloud](https://i.blackhat.com/asia-20/Friday/asia-20-Xu-Win-The-0-Day-Racing-Game-Against -Botnet-In-Public-Cloud.pdf)
- datacon 2020 Botnet Detection
dga domain name detection
Web Security Anomaly Detection
- LSTM identifies malicious HTTP requests
- Mini deployment of machine learning model based on URL anomaly detection
- My AI Security Detection Study Notes (1)
- Web attack classification and detection model based on machine learning
- Machine Learning Based Attack Detection System
- WAF Construction and Operation and AI Application Practice
- Advantages of Machine Learning in Web Security Detection
- [APT detection based on machine learning](https://mp.weixin.qq.com/s?__biz=MzU5MTM5MTQ2MA==&mid=2247484139&idx=1&sn=0da63a49f341eccc0bb48c954d8ebbb4&chksm=fe2efd60c95974767521fe6a6b7257a1d05e5482fc7ddeda281bdf0f0deb20add82d1a82d8ec&mpshare=1&scene=1&srcid=&pass_ticket=bjnNiDKomd79pQvRonW%2BXsTe6JrO%2FFs6oII12dZaLBPuQOtNK6Rzh9WSJ %2B%2F89ZUA#rd)
- RSAC 2019 | Machine Learning Algorithm Analysis Engine Helps Security Threat Reasoning Analysis
- Solving the last mile between machine learning and security operations
- RSAC 2019 | Using NLP Machine Learning for Automated Compliance Risk Management
Time Baseline
Getting Started with Penetration Testing
Wind control
- Shumei Risk Control
- Aliyun Artificial Intelligence waf
- Du Zhongwei: Identification and Traceability of Shell Black Ash Products
- How to build a good intelligent risk control tool system?
- Automated Iteration of Intelligent Risk Control Model
- Fourth Paradigm Intelligent Risk Control Middle Platform Architecture Design and Application
- 58 City Risk Control Platform Evolution
- Risk Control Modeling Process: Take the JD Group Perception Project as an Example
- Huya Risk Control
- Betta Fish
Security Conference Presentation Collection
data set
1、Samples of Security Related Dats
2、DARPA Intrusion Detection Data Sets
5、Data Capture from National Security Agency
6、The ADFA Intrusion Detection Data Sets
9、Multi-Source Cyber-Security Events
10、Malware Training Sets: A machine learning dataset for everyone
-
Vulnbank_dataset. A competition project of the KDD competition, the main purpose is to use machine learning methods to build an intrusion detector. The intrusion behaviors mainly include: DDOS, password brute force cracking, buffer overflow, scanning and other attack behaviors.
Excellent open source recommendation
- https://github.com/LiaoWenzhe
- https://github.com/yzhao062/pyod
- https://github.com/yzhao062/anomaly-detection-resources
- Machine Learning in Cybersecurity Collection
- The Ultimate Security Data Science and Machine Learning Guide
- Machine Learning for Cyber Security
- 404 Master's finishing
- Awesome-AI-Security
- awesome-ml-for-cybersecurity
- The Definitive Security Data Science and Machine Learning Guide
- https://github.com/0xMJ/AI-Security-Learning
- Dark Cloud
way of thinking:
- Propose good ideas and directions
- Liu Zhiyuan: Where do good research ideas come from
- MIT Artificial Intelligence Lab: How to do research
Utilities
- ReadPaper paper reading platform
- arxiv
- google scholar
- Baidu rasp security detection tool
Excellent public account
- Ali Security Emergency Response Center
- Tencent Security Emergency Response Center
- Baidu Security Emergency Response Center
- freebuf
- 先知社区
Related Top Clubs
- BlackHat / BlackHat Asia
- owasp
- botconf
- DEF-CON
- S&P
- CCS
- ICDFC
- USENIX Security
- PETS
- Wisec
- CODASPY
- ICSE
- NDSS
- Computer & Security
- TDSC
- RSAC
Related companies
- Omniscience Technology
- salt
- NSFOCUS
- Anheng Information
- Flash information
- Qi Anxin
Related events
- DataCon
- DataFountain
excellent books
- "Introduction to Machine Learning for Web Security"
- "Deep Learning in Web Security"
- "Reinforcement Learning and Gan of Web Security"
Related Blogs
- https://blog.csdn.net/Liao_Wenzhe/
- http://iami.xyz
- https://www.cdxy.me/
- Alibaba Cloud Security
Some interesting web attack and defense speeches on BlackHat
- https://www.blackhat.com/docs/asia-17/materials/asia-17-Dong-Beyond-The-Blacklists-Detecting-Malicious-URL-Through-Machine-Learning.pdf
- https://i.blackhat.com/briefings/asia/2018/asia-18-Simakov-Marina-Breaking-The-Attack-Graph.pdf
- https://i.blackhat.com/asia-19/Fri-March-29/bh-asia-Pham-Automated-REST-API-Endpoint.pdf
- https://i.blackhat.com/asia-20/Friday/asia-20-Hao-Attacking-And-Defending-Machine-Learning-Applications-Of-Public-Cloud.pdf
- https://i.blackhat.com/eu-19/Wednesday/eu-19-Kettle-HTTP-Desync-Attacks-Request-Smuggling-Reborn.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Gil-Web-Cache-Deception-Attack.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Burnett-Ichthyology-Phishing-As-A-Science-wp.pdf
- https://i.blackhat.com/us-18/Thu-August-9/us-18-Kettle-Practical-Web-Cache-Poisoning-Redefining-Unexploitable.pdf
- https://i.blackhat.com/us-18/Thu-August-9/us-18-Kettle-Practical-Web-Cache-Poisoning-Redefining-Unexploitable.pdf
- https://i.blackhat.com/USA-19/Wednesday/us-19-Valenta-Monsters-In-The-Middleboxes-Building-Tools-For-Detecting-HTTPS-Interception.pdf
- https://i.blackhat.com/USA-20/Wednesday/us-20-Kettle-Web-Cache-Entanglement-Novel-Pathways-To-Poisoning.pdf
- https://www.163.com/dy/article/GPJBLI020511CJ6O.html
- https://i.blackhat.com/USA-20/Wednesday/us-20-Klein-HTTP-Request-Smuggling-In-2020-New-Variants-New-Defenses-And-New-Challenges.pdf
- https://i.blackhat.com/EU-21/Wednesday/EU-21-Thatcher-Practical-HTTP-Header-Smuggling.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Gavrichenkov-Breaking-HTTPS-With-BGP-Hijacking-wp.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Sivakorn-HTTP-Cookie-Hijacking-In-The-Wild-Security-And-Privacy-Implications-wp.pdf
- https://towardsdatascience.com/deep-learning-for-specific-information-extraction-from-unstructured-texts-12c5b9dceada
- https://portswigger.net/research/cracking-the-lens-targeting-https-hidden-attack-surface
- https://www.botconf.eu/category/keynote/
- https://www.botconf.eu/2016/getting-your-hands-dirty-how-to-analyze-the-behavior-of-malware-traffic-and-web-connections/
- https://www.botconf.eu/2015/dga-clustering-and-analysis-mastering-modern-evolving-threats/
- https://www.blackhat.com/us-16/briefings/schedule/#account-jumping-post-infection-persistency--lateral-movement-in-aws-4309
- https://www.blackhat.com/us-16/briefings/schedule/#http-cookie-hijacking-in-the-wild-security-and-privacy-implications-3467
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Kettle-Cracking-The-Lens-Exploiting-HTTPs-Hidden-Attack-Surface.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Kettle-Cracking-The-Lens-Exploiting-HTTPs-Hidden-Attack-Surface-wp.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Prandl-PEIMA-Harnessing-Power-Laws-To-Detect-Malicious-Activities-From-Denial-Of-Service-To-Intrusion-Detection-Traffic-Analysis-And-Beyond.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Prandl-PEIMA-Harnessing-Power-Laws-To-Detect-Malicious-Activities-From-Denial-Of-Service-To-Intrusion-Detection-Traffic-Analysis-And-Beyond-wp.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Hypponen-The-Epocholypse-2038-Whats-In-Store-For-The-Next-20-Years.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Amiga-Account-Jumping-Post-Infection-Persistency-And-Lateral-Movement-In-AWS.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Sivakorn-HTTP-Cookie-Hijacking-In-The-Wild-Security-And-Privacy-Implications.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Gelernter-Timing-Attacks-Have-Never-Been-So-Practical-Advanced-Cross-Site-Search-Attacks.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Ermishkin-Viral-Video-Exploiting-Ssrf-In-Video-Converters.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Gavrichenkov-Breaking-HTTPS-With-BGP-Hijacking.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Zadeh-From-False-Positives-To-Actionable-Analysis-Behavioral-Intrusion-Detection-Machine-Learning-And-The-SOC.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Zadeh-From-False-Positives-To-Actionable-Analysis-Behavioral-Intrusion-Detection-Machine-Learning-And-The-SOC-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Morgan-Web-Timing-Attacks-Made-Practical.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Morgan-Web-Timing-Attacks-Made-Practical-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Saxe-Why-Security-Data-Science-Matters-And-How-Its-Different.pdf
- https://www.blackhat.com/docs/us-14/materials/us-14-Pinto-Secure-Because-Math-A-Deep-Dive-On-Machine-Learning-Based-Monitoring-WP.pdf
- https://www.blackhat.com/docs/us-14/materials/us-14-Pinto-Secure-Because-Math-A-Deep-Dive-On-Machine-Learning-Based-Monitoring.pdf
- https://media.blackhat.com/us-13/US-13-Pinto-Defending-Networks-with-Incomplete-Information-A-Machine-Learning-Approach-Slides.pdf
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2013/US-13-Pinto-Defending-Networks-with-Incomplete-Information-A-Machine-Learning-Approach-Slides.pdf
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2013/US-13-Peck-Abusing-Web-APIs-Through-Scripted-Android-Applications-WP.pdf
- https://www.youtube.com/watch?v=RGqCZO3cgY8
- https://www.youtube.com/watch?v=JUY4DQZ02o4
- https://www.youtube.com/watch?v=D6MG2uBIfUI
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2011/BH_US_11_Balduzzi_HPP_Slides.pdf
- https://www.blackhat.com/html/bh-us-11/bh-us-11-archives.html
- https://www.blackhat.com/docs/eu-14/materials/eu-14-Hafif-Reflected-File-Download-A-New-Web-Attack-Vector.pdf
- https://www.madlab.it/slides/BHEU2011/whitepaper-bhEU2011.pdf
- https://infocon.org/cons/Black%20Hat/Black%20Hat%20Europe/Black%20Hat%20Europe%202011/Presentations/Raul_Siles/BlackHat_EU_2011_Siles_SAP_Session-WP.pdf
- https://www.blackhat.com/presentations/bh-europe-09/Zanero_Criscione/BlackHat-Europe-2009-Zanero-Criscione-Masibty-Web-App-Firewall-slides.pdf
- https://infocon.org/cons/Black%20Hat/Black%20Hat%20USA/Black%20Hat%20USA%202007/presentations/Bolzoni_and_Zambon/Whitepaper/bh-usa-07-bolzoni_and_zambon-WP.pdf
- https://www.blackhat.com/html/bh-media-archives/bh-archives-2007.html#eu_07