|
Use AI to Detect AI-Generated Text (3)
特别鸣谢(Special thanks):在读论文的过程中,有几小点疑惑,所以当时请教了论文作者。非常非常感谢作者耐心又细致的答疑解惑,受益匪浅。While reading the paper, we had a few small questions, therefore we consulted the author. We are extremely grateful for the author's patient and detailed explanations, which were very enlightening.
如果需要这一系列或者其他文章的PPT可编辑源文件(免费),私信发送“获取”即可。To request the editable Slides (free) of this or other articles, send a private message with "Slides".
本文微信公众号版本: 用AI察觉AI生成的文本(3)验证检测能力的不同场景目录(Table of Contents):
<hr/>验证检测能力的不同场景(Testbed3~4)
在前文中我们提到了,为了试一试Detector检测AI生成文本的能力有多强,会为它准备不同难度的试炼关卡。这样的关卡一共有8个。
In the previous article, we mentioned that to test how well a detector can identify AI-generated text, it will be challenged with different difficulty levels. There are a total of eight such trial levels.
同时,我们也已经介绍了前2个比较容易的关卡,即Detector比较容易应对的场景(testbed1~2)。如果你已经忘了是什么场景,可以根据下面这两张图或者阅读前文再复习一下。
Meanwhile, we have also introduced the first two easier levels, which are scenarios that the Detector can handle more easily (testbed1~2). If you have forgotten what these scenarios are, you can refer to the two images below or read the previous post to review.
在这一篇中,我们会再介绍2个关卡,它们的难度也会越来越高喔 !
In this article, we will introduce two more levels, and they will become increasingly challenging!
Testbed 3
Seen: Domain1, Model1~7
在训练时,Detector已经熟悉(Seen)了属于Domain1的文本,也熟悉了由Model1~7生成的属于Domain1的文本。
During training, the Detector has already become familiar with texts from Domain1, as well as texts generated by Model1 to Model7 that belong to Domain1.
Unseen: /
在测试时,Detector要面临的挑战也是要识别属于Domain1的文本(其中AI生成的文本是由Model1~7生成的)。听起来是不是觉得这个关卡还不是很难?
During testing, the Detector’s challenge is also to identify texts from Domain1 (with AI-generated texts produced by Model1 to Model7). Doesn’t this level sound not too difficult?
Score:
为了能够更好的评价Detector在这种情景下的表现得分,我们当然要充分利用手中的数据。所以,要把每种可能的Domain都使用类似上述的实验设置(包括训练和测试阶段)跑一遍,从而可以得到10个得分(因为在数据集中,我们一共准备了10个Domain的数据)。最终,取平均分就是Detector在这道关卡下的最终得分了。
To better evaluate the Detector’s performance in this scenario, we need to make full use of the data at hand. Therefore, we should run the experiment setup (including both training and testing phases) for each possible Domain. This way, we can obtain 10 scores (since we have data for 10 Domains in the dataset). Finally, the average score will be the Detector’s final score for this level.
Testbed 4
Seen: Domain1~10, Model1~7
在训练时,Detector已经熟悉(Seen)了属于Domain1~10的文本,也熟悉了由Model1~7生成的属于Domain1~10的文本。
During training, the Detector has already become familiar with texts from Domain1 to Domain10, as well as texts generated by Model1 to Model7 that belong to Domain1 to Domain10.
Unseen: /
在测试时,Detector要面临的挑战也是要识别属于Domain1~10的文本(其中AI生成的文本是由Model1~7生成的)。
During testing, the Detector&#39;s challenge is also to identify texts from Domain1 to Domain10 (with AI-generated texts produced by Model1 to Model7).
Score:
这1个Detector在测试集上的分数就是它在这种情况下的最终得分。如果你不明白为什么这里只需要1个Detector,说明还没有很好的理解前3关的设定,强烈建议再看一下testbed1~3部分的解释。
The score of this Detector on the test set is its final score in this scenario. If you don&#39;t understand why only one Detector is needed here, it means you haven&#39;t fully grasped the setup of the first three scenarios. It&#39;s highly recommended to review the explanations in testbed1~3 sections.
总之,在这道关卡中,Detector学习了更多的内容(看过了更多的Domain的文本,也看过了更多的模型生成的文本),在测试时Detector面临的挑战也跟着丰富多样了。
In this scenario, the Detector has learned more (having seen texts from various domains and AI-generated texts from more models). Consequently, the challenges it faces during testing have also become more diverse.
小小的总结 Summary
通过关卡1~4,你应该也已经看出来了,尽管每一关的难度是逐渐变高的,但我们似乎并没有设置太高难度的关卡。为什么这么说呢?
Having gone through testbeds 1 to 4, you might have noticed that although each testbed gets progressively harder, we haven&#39;t really set any extremely difficult testbeds.
因为在测试Detector时,我们给它出的测试数据都是似曾相识的(来自相同的Domain,AI生成的文本来自相同的Model),所以Detector可能会觉得很得心应手。这种关卡的设定称作“In Distribution Settings”。
When testing the Detector, we provide it with familiar test data (from the same domains, with AI-generated text from the same models), which makes the Detector feel quite at ease. This type of scenario is called &#34;In Distribution Settings.&#34;
我相信你一定不会满足于这么轻易的就让Detector通关,在后面我们会介绍更难的关卡,即“Out-of-Distribution Settings”。
I believe you won&#39;t be satisfied with letting the Detector pass so easily. Later, we will introduce more challenging scenarios, known as &#34;Out-of-Distribution Settings.&#34;
(未完待续, To be continued)
小提醒:在公众号菜单模式,选择“所有文章”可以查看最新的所有文章列表,选择“版权声明”查看如何在其他场合使用此文章的内容。If you like the slides for this series or any other articles, please follow my wechat publich account and leave me message &#34;Slides&#34;. I understand you may not have a wechat account. Leaving messages via Github also works. To check the completed list of all the published articles (In English), please visit https://createmomo.github.io/
|
本帖子中包含更多资源
您需要 登录 才可以下载或查看,没有账号?立即注册
×
|