Generated Content Is Dangerous

Part 1: Malware Detection

At first we failed. The Sophos SOREL-20M dataset and accompanying models work on compiled binaries. The OpenAssistant InstructGPT pipe does not currently include a compilation pipe. Probably a good thing until we sort a few things out. Anyway, unless we want to code detect and dynamically compile output from the LLM system, this approach won’t work. Something for the future.

Yara

Yara is a pattern matching tool aimed at helping researchers identify and classify samples of data. From an LLM, the data is mostly, if not all, text. Yara is broader, though, in that it can be used to find patterns in any file or bytestream including images and buffers.

This is the first time I’d ever been exposed to the tool. It’s always amazing to find an entire ecosystem of developers, researchers, and practitioners I’d never really been exposed to before.

Goal: Dump the rules from Yara Project and get a simple backend up with the rules compiled that can quickly search a text string.

Get the code:

Content Scanner Repo: verbose-guacamole

Special Thanks

OpenAssistant - A super cool, potentially very empowering, project I am excited to be helping with. Check them out at open-assistant.io Yara - Documentation site Yara Rules Project - Contributed yara rules and helpful utilities. This group is quite good at regex.

Extra Images