{"sources":[{"id":"S1","url":"https://arxiv.org/html/1610.05820v2","title":"Membership Inference Attacks against Machine Learning Models","publisher":"arXiv; authors' paper for IEEE Symposium on Security and Privacy","published_at":"2017-03-31","accessed_at":"2026-11-15","version":"arXiv:1610.05820v2","document_id":"","revision_id":"","limitations":"Historical classifier study. Abstract and introduction support the definition and mechanism; service-specific results are not generalized. Version date checked at https://arxiv.org/abs/1610.05820."},{"id":"S2","url":"https://arxiv.org/html/2112.03570v2","title":"Membership Inference Attacks From First Principles","publisher":"arXiv","published_at":"2022-04-12","accessed_at":"2026-11-15","version":"arXiv:2112.03570v2","document_id":"","revision_id":"","limitations":"Section III-B supports evaluation recommendations, not the synthetic rates. No LiRA implementation or measured performance is claimed. Version date checked at https://arxiv.org/abs/2112.03570."}],"claims":[{"id":"C1","text":"Membership inference concerns a supplied record's training membership; prediction outputs can support black-box attacks.","location":"membership; questions Q1","evidence":[{"source_id":"S1","passage":"given a data record and black-box access","location":"Abstract"}],"limitations":"Possibility demonstrated for studied settings, not universal success or record extraction.","kind":"source","basis":"Fresh web reads of S1 abstract and introduction, particularly the membership objective and prediction-output discussion."},{"id":"C2","text":"Shadow models with known membership supply labeled outputs for training an attack classifier.","location":"membership","evidence":[{"source_id":"S1","passage":"train the attack model on the labeled inputs and outputs of the shadow models","location":"Section I, Introduction"}],"limitations":"High-level account of this method only.","kind":"source","basis":"Fresh read of introduction's consecutive shadow-model and attack-training paragraphs."},{"id":"C3","text":"Evaluate TPR at low FPR and report logarithmic ROC curves rather than aggregate accuracy alone.","location":"membership; reporting","evidence":[{"source_id":"S2","passage":"report an attack’s true-positive rate at low false-positive rates","location":"Section III-B, True-Positive Rate at Low False-Positive Rates"},{"source_id":"S2","passage":"always also report full (log-scale) ROC curves","location":"Section III-B, final paragraph"}],"limitations":"Methodological recommendation; no universal operating threshold is established.","kind":"source","basis":"Fresh read of S2 Section III-B low-FPR and ROC discussion."},{"id":"C4","text":"The notebook is an original synthetic counting exercise without a detector, model queries or personal records.","location":"membership scope paragraph","evidence":[],"limitations":"No empirical privacy conclusion; table selected for count comparison.","kind":"process","basis":"Inspectable content and lab.py use invented rates and arithmetic only; assets contain no model interface or diagram."},{"id":"C5","text":"The proposed assessment fixes record membership, model and threshold and separates development from assessment.","location":"experiment; reporting; questions Q1","evidence":[],"limitations":"Author's hypothetical design recommendation, not a performed experiment.","kind":"original_example","basis":"Explicit assumptions in experiment: one record per candidate, frozen training data, known membership labels, distinct development and assessment collections. Record and person counts are different units."},{"id":"C6","text":"TPR, FPR and precision use different denominators; FPR is not the error fraction among positive predictions.","location":"experiment; questions Q2","evidence":[],"limitations":"Nonzero denominators required.","kind":"original_example","basis":"Counting identities: members=TP+FN; nonmembers=FP+TN; predicted positives=TP+FP. Dividing the corresponding counts gives the stated definitions."},{"id":"C7","text":"The two constructed populations yield precision 98.36% and 37.74%, accuracy 79.50% and 98.61%, and a 99% always-negative baseline in the second.","location":"worked-example prose and table; code; questions Q2-Q3","evidence":[],"limitations":"Constructed counts, not detector measurements; displayed percentages rounded.","kind":"original_example","basis":"lab.py counts and main: TP=600 in both, FP=10 or 990, FN=400, TN=990 or 98010. Precision=600/610 or 600/1590; accuracy=1590/2000 or 98610/100000; baseline=99000/100000."},{"id":"C8","text":"Population precision is pt/[pt+(1-p)f]; at p=.01 and t=.60, 90% precision requires f<=1/1485 under the stated assumptions.","location":"worked-example; code; questions Q4","evidence":[],"limitations":"Positive denominator, fixed conditional rates and transferable populations assumed. Real threshold changes need not preserve TPR.","kind":"original_example","basis":"Original arithmetic from expected TP=Npt and FP=N(1-p)f. Solving precision>=q gives f<=pt(1-q)/(q(1-p)); lab.py sets p=1/100, t=3/5 and q=9/10. Rates are invented premises, not paper results."},{"id":"C9","text":"The illustrative synthetic lab performs rational arithmetic without shell, file, network or model calls; arithmetic execution is distinct from detector validation.","location":"lab; lab.py","evidence":[],"limitations":"Helpers assume valid inputs. This author turn did not execute Python. The supplied review reports a subsequent successful isolated CPython 3.12 run of the prior lab, whose code is retained; the raw execution transcript was not supplied here. That report supports only this synthetic calculation, not an implemented or validated detector.","kind":"process","basis":"Inspectable lab.py imports fractions.Fraction and contains arithmetic, loop, assertions and print. User-supplied source review C9 explicitly identifies a successful subsequent isolated run; C7 reports agreement with its output. The previous author-stage nonexecution statement is not treated as a statement about that later run. No claim that execution records do not exist is made."},{"id":"C10","text":"Empirical FPR increments are 1/n; observing zero false positives is compatible with a positive population rate.","location":"reporting; questions Q5","evidence":[],"limitations":"The zero-error probability assumes independent trials sharing fixed f; it is not an uncertainty interval.","kind":"original_example","basis":"1/100=1%; 1/10000=.01%. Under independent Bernoulli trials P(zero)=(1-f)^n; .999^100 is approximately .904792."},{"id":"C11","text":"Separate assessed counts from prevalence projections; failure of one detector alone does not establish other detectors' failure or a universal privacy guarantee.","location":"reporting","evidence":[],"limitations":"Narrow inference limit, not a claim that outcomes of different detectors can never be informative about one another.","kind":"original_example","basis":"The example changes prevalence while fixing rates and obtains different precision. Its stipulated single detector and arithmetic contain no premise establishing outcomes for all other detectors."},{"id":"C12","text":"Evidence scope and arithmetic scope are distinct; source reading is not independent expert verification.","location":"references source-check disclosure; bibliography metadata","evidence":[],"limitations":"No claim authenticates the previous attempt's author-stage tool history. Current access dates describe this turn's reads. The saved discovery excerpt is not cited as corroboration; S1 and S2 have no saved-original identifiers. No refined_work field was returned by the available public-original search interface.","kind":"process","basis":"Current-turn tool record: search_knowledge queries 'membership inference' and 'privacy attacks'; read_document for public discovery excerpt 1c18216b335b6ff919e3470195bb884e returned revision 09b7c9d43a9ec2ad5658328bd70e4b46. web opens read both versioned HTML papers and both arXiv abstract records; subsequent finds displayed S1 introduction and S2 Section III-B passages. The session date is 2026-11-15. These are current author-reported tool actions, not retrospective authentication of the earlier attempt. Attached visual references informed markup; no new rendering or browser interaction test is claimed."}]}