22 KiB
22 KiB
dataRisk-detection-resources(数据安全智能风控学习资源)
English | 简体中文
随着2021年中国《数据安全保护法》发布，意味着数据安全有望在中国形成新的风口.
笔者有幸在2021年加入中国领先的数据安全创业公司之一，从事数据科学结合数据安全的前沿研究和落地工作。在探索过程中发现网络上专门针对数据安全的资料不多，遂诞生了自己整理相关资料和思考的想法，希望尽自己的微薄之力推动社区发展。
拒绝白嫖，欢迎star！
一个人可以走得很快，只有一群人才能走得更远。笔者组建了个大数据安全技术交流的群，群友遍布硅谷，新加坡，腾讯，阿里，浙大等等，欢迎志同道合的朋友与我联系加入！
最近更新日期为：2022/11
AI应用防御篇
用AI来做应用安全防护
个人作品：数据安全智能风控落地实践
- isc2022主题演讲：AI驱动API安全风险检测与运营
- https://mp.weixin.qq.com/s/Ce8iXvAuNf2n3OFZSmFi1Q
- https://zhuanlan.zhihu.com/p/466955597?
- https://zhuanlan.zhihu.com/p/511095084
- https://mp.weixin.qq.com/s/Sme4gLnEHyxyhRSN2RUqCA
- https://www.zhihu.com/column/c_1471819989803700224
入门综述
- 为什么机器学习解决网络安全问题总是失败：谈谈特征空间
- 为什么机器学习解决网络安全问题总是失败：脆弱的系统工程
- 为什么机器学习解决网络安全问题总是失败：不合理的评估指标
- 为什么机器学习解决网络安全问题总是失败：机器学习不是万能灵药
- 求解网络安全问题的可解释机器学习
- https://github.com/LiaoWenzhe/BigdataAi
经典工业论文
OWASP10
API风险发现系统
- https://mp.weixin.qq.com/s/-9xkAROp7_A6gDjTLxfUsg
- https://www.freebuf.com/articles/web/189981.html
- https://search.freebuf.com/search/?search=%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0#article
- 基于机器学习的敏感信息泄露治理探索
风险业务
恶意注册账户
- 《Unveiling Fake Accounts at the Time of Registration: An Unsupervised Approach》
- 《DeepScan: Exploiting Deep Learning for Malicious Account Detection in Location-Based Social Networks》
恶意邮件
恶意流量检测
机器学习与安全
- 结合强化学习与CNN的Webshell检测方法
- 自动机器学习的安全风险
- 利用开源情报发现并解释恶意行为
- BadNL: 语义保持改进的NLP模型后门攻击
- APTMalInsight：基于系统调用信息和本体知识框架识别和认知APT恶意软件
图数据挖掘
- 《A Practical Approach to Constructing a Knowledge Graph for Cybersecurity》
- 《Developing an Ontology for Cyber Security Knowledge Graphs》
- 《Towards a Relation Extraction Framework for Cyber-Security Concepts》
- https://zhuanlan.zhihu.com/p/406415230
- https://zhuanlan.zhihu.com/p/69159780
- https://zhuanlan.zhihu.com/p/74274673
- https://zhuanlan.zhihu.com/p/75123819
marchine learning for UEBA
- 《AI2: Training a big data machine to defend》
- 《Big Data Security Challenges: An Overview and Application of User Behavior Analytics》
- 《Adaptive Intrusion Detection System via Online Learning》
- 《A multi-model approach to the detection of web-based attacks》
- 《McPAD : A Multiple Classifier System for Accurate Payload-based Anomaly Detection》
- 《Using Generalization and Characterization Techniques in the Anomaly-based Detection of Web Attacks》
- 《Anomaly-Based Web Attack Detection: A Deep Learning Approach》
- 《A Big Data Analysis Framework for Model-Based Web User Behavior Analytics》
- 《Anomalous Payload-based Network Intrusion Detection》
- 《Data mining for security at Google》
- 《User and Entity Behavior Analytics for Enterprise Security》
- 《A Comprehensive Approach to Intrusion Detection Alert Correlation》
- 《Trafc Anomaly Detection Using K-Means Clustering》
- 《Calculation of the Behavior Utility of a Network System: Conception and Principle》
- 《Spectrogram: A Mixture-of-Markov-Chains Model for Anomaly Detection in Web Traffic》
- 《用户画像相关技术》
MLOPS
入侵检测
- 基于机器学习的WEB攻击分类检测模型
- https://blog.cloudflare.com/api-abuse-detection/
- 利用机器学习检测HTTP恶意外连流量
- ExecScent: Mining for New C&C Domains in Live Networks with Adaptive Control Protocol Templates
- MADE: Security Analytics for Enterprise Threat Detection
- 机器学习在互联网巨头公司实践
- 机器学习在入侵检测方面的应用 - 基于ADFA-LD训练集训练入侵检测判别模型
- datacon比赛方向三-攻击源与攻击者分析writeup
- 基于机器学习的恶意软件加密流量检测研究分享
- anomaly-detection-through-reinforcement-learning
恶意url检测
- URLNet：通过深度学习学习URL表示以进行恶意URL检测
- 我的AI安全检测学习笔记（一）
- 《Compromised or Attacker-Owned: A Large Scale Classification and Study of Hosting Domains of Malicious URLs》
DDOS
- 基于KDDCUP 99数据集预测DDoS攻击
- 基于谱分析与统计机器学习的DDoS攻击检测技术研究
- 基于机器学习的分布式拒绝服务攻击检测方法研究
- DDoS Attacks Using Hidden Markov Models and Cooperative ReinforcementLearning*
僵尸网络检测
dga域名检测
Web安全异常检测
- LSTM识别恶意HTTP请求
- 基于URL异常检测的机器学习模型mini部署
- 我的AI安全检测学习笔记（一）
- 基于机器学习的WEB攻击分类检测模型
- 基于机器学习的攻击检测系统
- WAF建设运营及AI应用实践
- Web安全检测中机器学习的经验之谈
- APT detection based on machine learning
- RSAC 2019 | 机器学习算法分析引擎助力安全威胁推理分析
- 解决机器学习和安全运营之间的最后一公里问题
- RSAC 2019 | 采用NLP机器学习来进行自动化合规风险治理
时间基线
渗透测试入门
风控
- 数美风控
- 阿里云人工智能waf
- 杜中伟：贝壳黑灰产识别与溯源
- 如何构建好的智能风控工具体系？
- 智能风控模型的自动化迭代
- 第四范式智能风控中台架构设计及应用
- 58同城风控平台演进
- 风控建模流程：以京东群体感知项目为例
- 虎牙风控
- 斗鱼
数据集
1、Samples of Security Related Dats
2、DARPA Intrusion Detection Data Sets
5、Data Capture from National Security Agency
6、The ADFA Intrusion Detection Data Sets
9、Multi-Source Cyber-Security Events
10、Malware Training Sets: A machine learning dataset for everyone
-
Vulnbank_dataset. KDD大赛的一个竞赛项目，主要目的是使用机器学习得手段建立一个入侵检测器。其中的入侵行为主要包括：DDOS、密码暴力破解、缓冲区溢出、扫描等多种攻击行为。
优秀开源推荐
- https://github.com/LiaoWenzhe
- https://github.com/yzhao062/pyod
- https://github.com/yzhao062/anomaly-detection-resources
- 网络安全中机器学习大合集
- 最终安全数据科学和机器学习指南
- Machine Learning for Cyber Security
- 404师傅的整理
- Awesome-AI-Security
- awesome-ml-for-cybersecurity
- The Definitive Security Data Science and Machine Learning Guide
- https://github.com/0xMJ/AI-Security-Learning
- 乌云
- 安全会议演示集
- 先知社区
- cmu/facebook/tencent/ali/mayi fraud malicious
思维方式：
实用工具
- ReadPaper论文阅读平台
- arxiv
- google scholar
- 百度rasp安全检测工具
优秀公众号
- 阿里安全应急响应中心
- 腾讯安全应急响应中心
- 百度安全应急响应中心
- freebuf
- datafuntalk
相关顶会
- BlackHat / BlackHat Asia
- ISC
- first
- geekpwn
- poc
- fit
- cis
- tsec
- BCS
- KCON
- XCON
- CansecWest
- HITB
- owasp
- botconf
- DEF-CON
- S&P
- CCS
- ICDFC
- USENIX Security
- PETS
- Wisec
- CODASPY
- ICSE
- NDSS
- Computer & Security
- TDSC
- RSAC
相关公司
- 全知科技
- salt
- 绿盟科技
- 安恒信息
- 闪捷信息
- 奇安信
- impera
相关赛事
- DataCon
- DataFountain
优秀书籍
- 《web安全之机器学习入门》
- 《web安全之深度学习实战》
- 《web安全之强化学习与Gan》
相关博客
BlackHat 上一些有意思的web攻防演讲
- https://www.blackhat.com/docs/asia-17/materials/asia-17-Dong-Beyond-The-Blacklists-Detecting-Malicious-URL-Through-Machine-Learning.pdf
- https://i.blackhat.com/briefings/asia/2018/asia-18-Simakov-Marina-Breaking-The-Attack-Graph.pdf
- https://i.blackhat.com/asia-19/Fri-March-29/bh-asia-Pham-Automated-REST-API-Endpoint.pdf
- https://i.blackhat.com/asia-20/Friday/asia-20-Hao-Attacking-And-Defending-Machine-Learning-Applications-Of-Public-Cloud.pdf
- https://i.blackhat.com/eu-19/Wednesday/eu-19-Kettle-HTTP-Desync-Attacks-Request-Smuggling-Reborn.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Gil-Web-Cache-Deception-Attack.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Burnett-Ichthyology-Phishing-As-A-Science-wp.pdf
- https://i.blackhat.com/us-18/Thu-August-9/us-18-Kettle-Practical-Web-Cache-Poisoning-Redefining-Unexploitable.pdf
- https://i.blackhat.com/us-18/Thu-August-9/us-18-Kettle-Practical-Web-Cache-Poisoning-Redefining-Unexploitable.pdf
- https://i.blackhat.com/USA-19/Wednesday/us-19-Valenta-Monsters-In-The-Middleboxes-Building-Tools-For-Detecting-HTTPS-Interception.pdf
- https://i.blackhat.com/USA-20/Wednesday/us-20-Kettle-Web-Cache-Entanglement-Novel-Pathways-To-Poisoning.pdf
- https://www.163.com/dy/article/GPJBLI020511CJ6O.html
- https://i.blackhat.com/USA-20/Wednesday/us-20-Klein-HTTP-Request-Smuggling-In-2020-New-Variants-New-Defenses-And-New-Challenges.pdf
- https://i.blackhat.com/EU-21/Wednesday/EU-21-Thatcher-Practical-HTTP-Header-Smuggling.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Gavrichenkov-Breaking-HTTPS-With-BGP-Hijacking-wp.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Sivakorn-HTTP-Cookie-Hijacking-In-The-Wild-Security-And-Privacy-Implications-wp.pdf
- https://towardsdatascience.com/deep-learning-for-specific-information-extraction-from-unstructured-texts-12c5b9dceada
- https://portswigger.net/research/cracking-the-lens-targeting-https-hidden-attack-surface
- https://www.botconf.eu/category/keynote/
- https://www.botconf.eu/2016/getting-your-hands-dirty-how-to-analyze-the-behavior-of-malware-traffic-and-web-connections/
- https://www.botconf.eu/2015/dga-clustering-and-analysis-mastering-modern-evolving-threats/
- https://www.blackhat.com/us-16/briefings/schedule/#account-jumping-post-infection-persistency--lateral-movement-in-aws-4309
- https://www.blackhat.com/us-16/briefings/schedule/#http-cookie-hijacking-in-the-wild-security-and-privacy-implications-3467
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Kettle-Cracking-The-Lens-Exploiting-HTTPs-Hidden-Attack-Surface.pdf
- https://www.blackhat.com/docs/us-17/wednesday/us-17-Kettle-Cracking-The-Lens-Exploiting-HTTPs-Hidden-Attack-Surface-wp.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Prandl-PEIMA-Harnessing-Power-Laws-To-Detect-Malicious-Activities-From-Denial-Of-Service-To-Intrusion-Detection-Traffic-Analysis-And-Beyond.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Prandl-PEIMA-Harnessing-Power-Laws-To-Detect-Malicious-Activities-From-Denial-Of-Service-To-Intrusion-Detection-Traffic-Analysis-And-Beyond-wp.pdf
- https://www.blackhat.com/docs/us-17/thursday/us-17-Hypponen-The-Epocholypse-2038-Whats-In-Store-For-The-Next-20-Years.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Amiga-Account-Jumping-Post-Infection-Persistency-And-Lateral-Movement-In-AWS.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Sivakorn-HTTP-Cookie-Hijacking-In-The-Wild-Security-And-Privacy-Implications.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Gelernter-Timing-Attacks-Have-Never-Been-So-Practical-Advanced-Cross-Site-Search-Attacks.pdf
- https://www.blackhat.com/docs/us-16/materials/us-16-Ermishkin-Viral-Video-Exploiting-Ssrf-In-Video-Converters.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Gavrichenkov-Breaking-HTTPS-With-BGP-Hijacking.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Zadeh-From-False-Positives-To-Actionable-Analysis-Behavioral-Intrusion-Detection-Machine-Learning-And-The-SOC.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Zadeh-From-False-Positives-To-Actionable-Analysis-Behavioral-Intrusion-Detection-Machine-Learning-And-The-SOC-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Morgan-Web-Timing-Attacks-Made-Practical.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Morgan-Web-Timing-Attacks-Made-Practical-wp.pdf
- https://www.blackhat.com/docs/us-15/materials/us-15-Saxe-Why-Security-Data-Science-Matters-And-How-Its-Different.pdf
- https://www.blackhat.com/docs/us-14/materials/us-14-Pinto-Secure-Because-Math-A-Deep-Dive-On-Machine-Learning-Based-Monitoring-WP.pdf
- https://www.blackhat.com/docs/us-14/materials/us-14-Pinto-Secure-Because-Math-A-Deep-Dive-On-Machine-Learning-Based-Monitoring.pdf
- https://media.blackhat.com/us-13/US-13-Pinto-Defending-Networks-with-Incomplete-Information-A-Machine-Learning-Approach-Slides.pdf
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2013/US-13-Pinto-Defending-Networks-with-Incomplete-Information-A-Machine-Learning-Approach-Slides.pdf
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2013/US-13-Peck-Abusing-Web-APIs-Through-Scripted-Android-Applications-WP.pdf
- https://www.youtube.com/watch?v=RGqCZO3cgY8
- https://www.youtube.com/watch?v=JUY4DQZ02o4
- https://www.youtube.com/watch?v=D6MG2uBIfUI
- https://paper.bobylive.com/Meeting_Papers/BlackHat/USA-2011/BH_US_11_Balduzzi_HPP_Slides.pdf
- https://www.blackhat.com/html/bh-us-11/bh-us-11-archives.html
- https://www.blackhat.com/docs/eu-14/materials/eu-14-Hafif-Reflected-File-Download-A-New-Web-Attack-Vector.pdf
- https://www.madlab.it/slides/BHEU2011/whitepaper-bhEU2011.pdf
- https://infocon.org/cons/Black%20Hat/Black%20Hat%20Europe/Black%20Hat%20Europe%202011/Presentations/Raul_Siles/BlackHat_EU_2011_Siles_SAP_Session-WP.pdf
- https://www.blackhat.com/presentations/bh-europe-09/Zanero_Criscione/BlackHat-Europe-2009-Zanero-Criscione-Masibty-Web-App-Firewall-slides.pdf
- https://infocon.org/cons/Black%20Hat/Black%20Hat%20USA/Black%20Hat%20USA%202007/presentations/Bolzoni_and_Zambon/Whitepaper/bh-usa-07-bolzoni_and_zambon-WP.pdf
- https://www.blackhat.com/html/bh-media-archives/bh-archives-2007.html#eu_07