So last week at offensivecon I watched a talk on Fuzzilli ( https://github.com/googleprojectzero/fuzzilli) which, I have to admit I had no idea what it was. Obviously I knew it was a Googlely Javascript fuzzer, finding bugs. But I did not realize that it was applying mutations to its own intermediate language which it then compiled to Javascript. I just assumed it was, like most fuzzers, mutating the javascript directly (f.e. https://sean.heelan.io/2016/04/26/fuzzing-language-interpreters-using-regression-tests/ ). But having an IL designed for fuzzing-related mutations is clearly a great idea! And this year, they've expanded on that to build a Javascript->Fuzzilli compiler/translation layer. So you can pass in sample Javascript and then it will create the IL and then it will mutate the IL. The reason this is necessary is that Javascript is. like almost all modern languages, extremely complicated underneath the covers, so in order to generate crashes you may need to have a lot of different fields set properly in a particular order in a structure. They try to do some introspection on objects and generate their samples from that as well, but there's no beating "real user code" for learning how an object needs to be created and used. These advances generate a lot more bugs! In theory none of these bugs matter in the future because of the mitigations (no pointers outside the Javascript gigacage!) going into place by the very authors of the fuzzer? (I have my doubts, but we all will live and learn?) It would be...very cool, I think, if Bard or another LLM was the one doing the Javascript sample generation as well. If you think about it, these LLMs all have a good understanding of Javascript and you can give them various weird tasks to do, and let them generate your samples, and then when a crash happens you can have them mutate around that crash, or if you have a sample not getting any more code coverage you can have them mutate that sample to attempt to make it weirder. :) -dave

IMO, we're in the toddler phase of LLMs right now. You know how toddlers put everything in their mouth? Since we're still figuring out what LLMs are good at, we're going to keep throwing weird stuff at them to see what happens. On Mon, May 22, 2023 at 10:04 AM Dave Aitel via Dailydave < dailydave@lists.aitelfoundation.org> wrote: > So last week at offensivecon I watched a talk on Fuzzilli ( > https://github.com/googleprojectzero/fuzzilli) which, I have to admit I > had no idea what it was. Obviously I knew it was a Googlely Javascript > fuzzer, finding bugs. But I did not realize that it was applying mutations > to its own intermediate language which it then compiled to Javascript. I > just assumed it was, like most fuzzers, mutating the javascript directly > (f.e. > https://sean.heelan.io/2016/04/26/fuzzing-language-interpreters-using-regression-tests/ > ). > > But having an IL designed for fuzzing-related mutations is clearly a great > idea! And this year, they've expanded on that to build a > Javascript->Fuzzilli compiler/translation layer. So you can pass in sample > Javascript and then it will create the IL and then it will mutate the IL. > The reason this is necessary is that Javascript is. like almost all modern > languages, extremely complicated underneath the covers, so in order to > generate crashes you may need to have a lot of different fields set > properly in a particular order in a structure. They try to do some > introspection on objects and generate their samples from that as well, but > there's no beating "real user code" for learning how an object needs to be > created and used. > > These advances generate a lot more bugs! In theory none of these bugs > matter in the future because of the mitigations (no pointers outside the > Javascript gigacage!) going into place by the very authors of the fuzzer? > (I have my doubts, but we all will live and learn?) > > It would be...very cool, I think, if Bard or another LLM was the one doing > the Javascript sample generation as well. If you think about it, these LLMs > all have a good understanding of Javascript and you can give them various > weird tasks to do, and let them generate your samples, and then when a > crash happens you can have them mutate around that crash, or if you have a > sample not getting any more code coverage you can have them mutate that > sample to attempt to make it weirder. :) > > -dave > > > > > _______________________________________________ > Dailydave mailing list -- dailydave@lists.aitelfoundation.org > To unsubscribe send an email to dailydave-leave@lists.aitelfoundation.org >