"""Section 16.4: differences in Saunders et al.'s early ablation of the number of critiques shown. The three rows are copied from Saunders et al. (2022), arXiv 2206.05802, appendix E.3, which the paper describes as from earlier in the project. "Critiques found" counts the critiques labelers wrote; it does not separate valid from invalid ones. Only the differences below are computed here. """ from decimal import Decimal as D rows = { # condition: (minutes, critiques found) "no assistance": (D("6.27"), D("0.74")), "4 critiques": (D("5.82"), D("1.06")), "8 critiques": (D("6.26"), D("1.31")), } base_t, base_n = rows["no assistance"] for name in ("4 critiques", "8 critiques"): t, n = rows[name] print(f"{name}: time {t - base_t:+} min, critiques found {n - base_n:+}") d4 = rows["4 critiques"][1] - base_n d8 = rows["8 critiques"][1] - base_n print(f"increase with 8 critiques / increase with 4 critiques = {d8}/{d4} = {d8 / d4:.2f}")