How to best mask stylometric characteristics in practice, and what tasks to perform manually, what with tool assistance, and what fully automatically, is an open field of research, especially in short documents with limited potential variability. Manual adversarial stylometry can be preferred or even required if the author does not trust available computers with the task (as may be the case for a whistleblower, for example). Software tools require
maintenance; report that there is no maintained obfuscatory software suitable for general use. identify DS-PAN and Mutant-X as the 2022 state of the art in automated obfuscation. Manual stylistic modulation is a significant effort, with poor
scalability properties; tool assistance can reduce the burden to varying degrees. Deterministic automated methods can lose effectiveness against a classifier
trained adversarially, where output from the style transfer program is used in the classifier's training set. give three criteria for use in evaluation of adversarial stylometry methods:
safety, meaning that stylistic characteristics are reliably eliminated;
soundness, meaning that the semantic content of the text is not unacceptably altered; and
sensible, meaning that the output is "well-formed and inconspicuous". Compromising any too deeply is typically an unacceptable result, and the three trade off against each other in practice. find that automatically evaluating sensibility, and specifically whether output is acceptably grammatical and well-formed, is difficult; automated evaluation of soundness is somewhat more promising, but manual review is the best method. Despite safety being an important property of an adversarial stylometry method, it can still be usefully traded away if the conceded stylometric identification potential is otherwise possible by non-stylometric analysis—for example, an author discussing their own upbringing in Britain is unlikely to care if stylometry can reveal that their text is typical of
British English. Evaluating the safety of different approaches is complicated by how identification-resistance fundamentally depends on the methods of identification under consideration. The property of being resilient to unknown analyses is called
transferability. identify four different
threat models for authors, varying with their knowledge of how their text will be analysed and what
training data will be used:
query access, with the weakest analyst and the strongest author who knows both the methods of analysis and the training data;
architecture access, where the author knows the analysis methods but not the training data;
data access, where the author knows the training data but not the analysis methods; and
surrogate access, with the weakest author and the strongest analyst, where the author does not know the methods of analysis nor the training data. Further, when an author chooses a method, they must rely on their threat model and trust that it is valid, and that unknown analyses able to detect remaining stylistic signals cannot or will not be performed, or that the masking successfully transfers; a stylometrist with knowledge of how the author attempted to mask their style, however, may be able to exploit some weakness in the method and render it unsafe. Much of the research into automated methods has assumed that the author has query access, which may not generalise to other settings. Masking methods that internally use an
ensemble of different analyses as a model for its adversary may transfer better against unseen analyses. A thorough soundness loss defeats the purpose of communication, though some degree of meaning change may be tolerable if the core message is preserved; requiring only
textual entailment or allowing
automatic summarisation are other options to lose some meaning in a possibly-tolerable way. Rewriting an input text to defeat stylometry, as opposed to consciously removing stylistic characteristics during composition, poses challenges in retaining textual meaning. assess the problem of unsoundness as "the most important challenge" for research into fully automatic approaches. For sensibility, if a text is so ungrammatical as to be incomprehensible or so ill-formed that it cannot fit in to its genre then the method has failed, but compromises short of that point may be useful. If inconspicuity is partially lost, then there is the possibility that more expensive and less scalable analyses will be performed (e.g., consulting a forensic linguist) to confirm suspicions or gather further evidence. The impact of a total inconspicuity failure varies depending on the motivation for performing adversarial stylometry: for someone simply attempting to stay anonymous (e.g., a whistleblower), detection may not be an issue; for a literary forger, however, detection would be disastrous. Adversarial stylometry can leave evidence of its practice, which is an inconspicuity failure. In the Brennan–Greenstadt corpus, the texts have been found to share a common "style" of their own. However, assess existing evidence as insufficient to prove that adversarial stylometry is always detectable, with only limited methods having been studied. Improving the smoothness of the output text may reduce the detectability of automated tools. The overall detectability of adversarial authorship has not been thoroughly studied; if the methods available to be used by the author are unknown to the stylometrist, it may be impossible. The problems of author identification and verification in an adversarial setting are greatly different from recognising naïve or cooperative authors. Deliberate attempts to mask authorship are described by as a "problem for the current state of stylometric art", and state that, despite stylometry's high performance in identifying non-adversarial authors, manual application of adversarial methods render it unreliable. observe that low-dimensional stylometric models which operate on small numbers of features are less resistant to adversarial stylometry. Research has found that authors vary in how well they are able to modulate their style, with some able to successfully perform the task even without training. , a replication and reproduction of , found that all three of imitation, translation and obfuscation meaningfully reduced the effectiveness of authorship attribution, with manual obfuscation being somewhat more effective than manual imitation or translation, which performed similarly to each other; the original study found that imitation was superior. reported that even simple automated methods of adversarial stylometry caused major difficulties for state-of-the-art authorship identification systems, though at significant soundness and sensibility cost. Adversarially-aware identification systems can perform much better against adversarial stylometry provided that they know which potential obfuscation methods were used, even if the identifier makes mistakes in analysing which anonymisation method was used. == See also ==