metrology lab experiments

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Lab Manual for Metrology and Machine tools

In this lab course, students can understand different machining processes through laboratory experiments. The students must have a fairly good understanding of the theory underlying the experiments before doing the experiments. The students can perform various tests such as 1. Use of gear teeth vernier callipers for checking the chordal addendum, 2. Machine tool alignment of test on the lathe, 3. Toolmakers microscope and its application, 4. Angle and taper measurement by bevel protractor and sine bar, 5. Use of spirit level and optical flats in ?finding the the flatness of the surface plate, 6. Thread measurement by 2-wire and 3-wire methods, 7. Introduction to general-purpose machine tools- lathe, drilling,milling and shaper machines, 8. Step Turning and aper turning on lathe machine, 9. Thread cutting and knurling on lathe machine, 10. Drilling and tapping operations

Related Papers

Prashanth Prashi

rajendra singh

surya selva

Hector Socrates Aguirre Caballero

Ashman Noordin

heru nugroho

penjelasan tentang besi

Cardo Dalisay

suresh karnan

Overview: This note set is part of a larger collection of materials available at http://claymore.engi-neer.gvsu.edu. You are welcome to use the material under the license provided at http://clay-more.engineer.gvsu.edu/eod/global/copyrght.html. As always any feedback you can provide will be welcomed.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

hemanth phani

Joshua Ade-Omowaye

Robert Zanfir

puni venkat

kamal touileb

Workshop technology

Shahadat Ali

Vishnu Chandran

Eduardo Henrique

Firoz Kumar

Raphael Silva Augusto

sokolyk sashko

Charles Conconi

Louis Antwi

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

Measuring measurement – What is metrology and why does it matter?

Richard j.c. brown.

a National Physical Laboratory, Teddington TW11 0LW, UK

• • A robust quality infrastructure is essential to address global challenges effectively.
• • Metrology is the, often invisible, infra-technology supporting the quality infrastructure.
• • Metrology remains a very small endeavour compared to all the activities that rely on it.
• • The concept of metrology as ‘measuring measurement’ is proposed.
• • This emphasises the meta-thought involved that distinguishes it from routine measurement.

Metrology remains a uniquely important endeavour. A sign of its success and robustness as an infra-technology is that it usually goes unnoticed. This means that it is in danger of being under-valued and under-appreciated. The sure-footing that metrology provides to the quality infrastructure will be especially important as the world grapples with the aftereffects of the COVID-19 pandemic, rebuilding global economies and also re-focusing on addressing global grand challenges and exploiting emerging technologies. In this context it is important and timely to re-examine the concept of metrology and how it relates to the quality infrastructure that it serves, but differs to measurement in general. The concept of metrology as ‘measuring measurement’ is proposed, emphasising the characteristic meta-thought associated with the discipline that distinguishes it from routine measurement.

1. Introduction

The International Bureau of Weight and Measures states that: “Metrology is the science of measurement, embracing both experimental and theoretical determinations at any level of uncertainty in any field of science and technology.” [1] Metrology is surely at the core of all practical scientific endeavours. Metrology is important because almost all of everyday life, not to mention practical science, technology, engineering and medicine, involves measurements that we rely on for our health, commercial prosperity, quality of life and the protection of the environment. Metrology is the structure that ensures these measurements are stable, comparable and accurate, providing confidence in measurement at a stated level (usually by quoting a measurement uncertainty). When these qualities are associated with a measurement it reduces waste, allows trade, enables infrastructure to function, technology to advance, the economy to prosper, encourages global agreement, collaboration and trade, and ensures our ongoing health and safety and quality of life. In short, metrology generates systems and frameworks for quantification and through these underpins consistency and assurance in all measurement. However, metrology, by most definitions, remains a very small endeavour compared to all the activities that rely on it. It is worth exploring this distinction in more detail.

2. Discussion

It is clear from its definition that the reach of metrology is far and wide. However, the term is not widely used or known. This is because metrology is an infra-technology. In the same way that infrastructures support society in general, infra-technologies support the operation of scientific and technical goals. If the making of measurements and exchange of measurement results is like traffic, then metrology is like the road network: a fundamental part of the system allowing for smooth and seamless operation of the whole. When working well it is unnoticed: when potholes appear on the road, we notice.

‘Measurement space’ in science and technology is huge – it is difficult to conceive of any scientific, technological, engineering or medical endeavour that doesn’t rely on measurement – but the part of this that is formally metrology is small. All of metrology may be considered as a subset of measurement, but not all of measurement is metrology – not by a long way. Even though many of the fundamental concepts of good measurement and metrology are similar – traceability, comparison and uncertainty – the vast majority of measurement sits outside metrology. Metrology, as the ‘science of measurement’ implies a meta-subject with a deeper level of abstraction. Metrology might be thought of as ‘meta-measurement’, that is: ‘measurement’ operating at a higher level in order to elaborate the properties of ‘normal’ measurement. (In the same way, in linguistics, grammar is considered as a metalanguage used to describe the properties of plain language but not itself.) To generalise, in the same way philosophy is known popularly as ‘thinking about thinking’ so we can consider metrology as ‘measuring measurement’.

Better measurement improves understanding of science and technology. Better metrology improves understanding of measurement. Improvements in science and technology enable advances in metrology. This is the virtuous circle that drives human development. By measurement, ‘good measurement’ is meant. That is: measurement exhibiting the qualities of stability, comparability (and coherence if traceable to the SI), with demonstrable, fit-for-purpose accuracy and a statement of uncertainty in the measurement values produced, under the control of a local quality system, and within the broader remit of a national quality infrastructure providing universal confidence in measurement. By contrast, poor measurement, not exhibiting these qualities, is not useful in advancing science.

Rather like most definitions there is no sharp delineation between what is measurement and what constitutes metrology. It is part of the same continuum ensuring confidence in measurement. We can consider work that is unique to the role of National Metrology Institutes (NMIs) and Designated Institutes (DIs) as clearly metrology. These activities are:

• • the definition and realisation of measurement units (note that these need not be SI units, but must be the highest point of reference nationally);
• • the maintenance, development and improvement of these primary national standards and realisations, and;
• • the dissemination of traceability and traceable measurement directly resulting from these primary standard realisations.

This is considered as ‘scientific metrology’ (or sometimes ‘fundamental metrology’) – in particular these are areas where global agreement is required [2] . We may also consider as metrology work to measure fundamental constants (often done in collaboration with university-based academics and occasionally by academics alone, for instance in measurement of big G in [3] ).

Assuring confidence in measurement results via certification, standardization, accreditation and calibration, often known as ‘industrial metrology’ [2] (since that is the sector where it is usually found) or ‘applied metrology’ may to a lesser extent be considered as containing aspects of metrology, but may also just be ‘good measurement’. The degree to which these areas can be considered metrology depends on the extent of meta-study and meta-thought that is involved at each step. For example, calibrating an instrument is not in and of itself metrology unless extra detailed thought is applied, for example: original contributions to the calculation and propagation of uncertainty of the measurement. It is clear that maintaining quality assured, accreditated measurement capability requires significant care and skill, but this doesn’t necessarily involve the detailed meta-thought and analysis required of metrology. Good measurement may still be routine measurement – there is nothing wrong with that.

Legal metrology [2] exists to deal with the application of the law to metrology and regulated measurement. In many ways it can be considered as infrastructural but at a more applied science level that the infra-technology of scientific metrology, often with local implementation depending on national jurisdiction. To extend the analogy above: if scientific metrology is the road network and measurement is traffic flow then legal metrology is the road signs, speed limits and traffic lights. Legal metrology has an important part to play in better regulation (in the analogy: keeping the traffic moving smoothly and safely and without congestion).

Meta-thought is more obvious and more concentrated in the activities of primary metrology at the top of the quality infrastructure. The metrology component becomes less clear and obvious and more disperse as one descends the, nonetheless essential, quality infrastructure components of accreditation, standardisation, conformity assessment and market surveillance [4] . By the time end user measurements are made, be they in the factory or in the field, the data produced is usually the endpoint in the traceability chain, and the mechanism of production is much more rarely subject to additional meta-thought. Indeed, in the majority of cases, measurement is not metrology, even good measurement. In particular, making measurements with no associated meta-thought is not metrology. Nor is metrology necessarily concerned with research into new measurement techniques, new instruments, applied science, curiosity driven research or discovery science, unless there is any associated or subsequent meta-thought, meta-study, or a clear linkage of these activities back to provision of primary standards or realisation of measurement units.

3. Conclusion

The world needs good measurement more than ever to ensure sustainable progress in science and in society. Good measurement enables societal progress and economic growth via an agreed quality infrastructure. Metrology sits at the top of this quality infrastructure providing the stable definitions of measurement units and realisation of primary standards for these, from which traceability chains can flow; and improving these realisations to continue to reduce uncertainties for the end user. Because of this underpinning, infrastructural characteristic, metrology and its role in the quality infrastructure often goes unnoticed, and its distinction from measurement can become blurred. Metrology is distinguished from other quality infrastructure components as it is based almost entirely on meta-thought and meta-measurement – ‘measuring measurement’ – a concept that is necessarily decreasingly present as one descends the quality infrastructure and traceability chain. Good measurement is necessary to make progress in science but not on its own sufficient without metrology accompanying it. A joined-up measurement infrastructure and quality system [5] :

• • Improves effectiveness of research and development and the trust in its outcomes;
• • Improves efficiency, reducing waste and increasing productivity;
• • Accelerates innovation, getting products to market faster, or allowing them to fail fast;
• • Enables change to happen faster, adding value to society;
• • Is essential for the development and assessment of evidence-based policy.

The world needs to realise these benefits now in order to address the challenges of COVID-19, stimulate economic recovery and return to facing global grand challenges such as: climate change, clean growth, achieving net zero carbon emissions by 2050, and an ageing population; and to exploiting emerging technologies such as: big data and the digital world, artificial intelligence, quantum technologies, industry 4.0, future communications and personalised medicine. These benefits of good measurement are particularly relevant for areas that don’t currently embrace the quality infrastructure, such as much of the academic sector and many small and medium-sized enterprises [6] , [7] .

We should celebrate metrology and the role it plays in keeping the measurement highway free of potholes, and better promote its benefits together with those of a joined-up quality infrastructure.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The funding of the National Measurement System by the UK Department for Business, Energy & Industrial Strategy is gratefully acknowledged.

Calibration and Verification of Measuring Instruments: Conceptual Transformation

• GENERAL PROBLEMS OF METROLOGY AND MEASUREMENT TECHNIQUE
• Published: 31 March 2022
• Volume 64 , pages 871–882, ( 2022 )

Cite this article

• S. F. Levin 1  

98 Accesses

Explore all metrics

The present article discusses problems associated with the calibration of measuring instruments (MIs). The author gives a brief review of the 1978–1992 discussion on problems related to the applicability of statistical methods, along with providing an insight into the transformation of calibration concepts according to regulatory documents. The general measurement problem of testing for type approval purposes, as well as MI verification and calibration, is considered as an issue concerning the identification of functional metrological characteristics. It is noted that MI calibration for operating conditions is of great importance during verification under normal conditions. The main difficulty in solving this problem is found to be the multidimensionality of the calibration function model and the need to take into account inadequacy-related errors along with measurement errors. It is shown that if only one of these components is factored in, the model may become unnecessarily complex with a loss of stability, or it could lead to the use of incomplete models of the calibration function giving the impression of high accuracy. Finally, the author provides an example of MI calibration for operating conditions using two influence quantities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Problems Concerning the Calibration of Measuring Instruments under Specified Conditions

The Problem of Determining and Adjusting the Inter-Calibration Intervals of Measuring Instruments

Comparison of Identification Criteria for Mathematical Models in Solving Measurement Problems

B. M. Gavrilov et al., Development of Activities on Measuring Instrument Calibration: Report of the Working Group of the Intersectoral Council for Applied Metrology and Instrumentation of the Russian Union of Industrialists and Entrepreneurs , A. S. Krivov (ed.), RSPP, Moscow (2016).

International Vocabulary of Metrology: Basic and General Concepts and Associated Terms (VIM) [Russian translation], NPO Professional, St. Petersburg (2010), 3rd ed.

Guide to the Expression of Uncertainty in Measurement [Russian translation], V. A. Slaev (ed.), VNIIM, St. Petersburg (1999).

A. N. Kolmogorov, “True meaning of analysis results,” in: Proc. 2nd All-Union Conf. on Math. Stat. , Izd. AN UzSSSR, Tashkent (1949), pp. 240–268.

Yu. I. Alimov, “Practical value of the estimation theory,” Avtomatika , No. 2, 84–94 (1981).

A. N. Kolmogorov, “Random number tables,” Semiot. Informat. , No. 18, 3–13 (1982).

Issues of Cybernetics. VK-94. Statistical Methods in the Theory of Operational Support , Scientific Council on the complex problem “Cybernetics,” AN SSSR, Moscow (1982).

Yu. P. Adler, Yu. V. Granovskii, and E. V. Markova, Theory of Experiment: Past, Present, and Future , Znanie (1982).

P. V. Novitskii and I. A. Zograf, Estimation of Measurement Errors , Energoatomizdat, Leningrad (1985).

Google Scholar  

S. F. Levin and A. P. Blinov, “Scientific and methodological support for the guaranteed solution of metrological problems using probabilistic and statistical methods,” Izmer. Tekhn. , No. 12, 5–8 (1988).

Statistical Identification, Prediction, and Control in Electronic Equipment: Guidelines , MO SSSR, Moscow (1990).

F. Mosteller and J. Tukey, Data Analysis and Regression [Russian translation], Finansy i Statistika, Moscow (1982), Iss. 2.

P. Huber, Robust Statistics [Russian translation], Mir, Moscow (1984).

I. Vuchkov, L. Boyadzhieva, and E. Solakov, Applied Linear Regression Analysis [Russian translation], Finansy i Statistika, Moscow (1987).

F. Hampel, E. Ronchetti, P. Rousseeuw, and W. Stahel, Robust Statistics: The Approach Based on Influence Functions [Russian translation], Mir, Moscow (1989).

MATH   Google Scholar  

A. N. Kolmogorov, “Logical basis of the probability theory,” in: Probability Theory and Mathematical Statistics: Collected Papers , Nauka, Moscow (1986).

S. F. Levin, “Metrology. Mathematical statistics. Legends and myths of the 20th century: the legend of uncertainty,” Partn. Konkur. , No. 1, 13–25 (2001).

H. Scheffe, The Analysis of Variance [Russian translation], FIZMATLIT, Moscow (1963).

S. F. Levin, “Guide to the Expression of Uncertainty in Measurement. Revision – paradigm shift or new sanction?” Zakonodat. Prikl. Metrol. , No. 5, 31–44 (2016).

S. F. Levin, “Measurement problem of calibrating a measuring instrument under specified conditions,” Izmer. Tekhn. , No. 4, 9–15 (2021), https://doi.org/ https://doi.org/10.32446/0368-1025it.2021-4-9-15 .

Download references

Author information

Authors and affiliations.

Moscow Institute for Expertise and Tests, Moscow, Russia

S. F. Levin

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to S. F. Levin .

Additional information

Translated from Izmeritel’naya Tekhnika, No. 11, pp. 8–18, November, 2021.

Rights and permissions

Reprints and permissions

About this article

Levin, S.F. Calibration and Verification of Measuring Instruments: Conceptual Transformation. Meas Tech 64 , 871–882 (2022). https://doi.org/10.1007/s11018-022-02017-4

Download citation

Received : 29 April 2021

Accepted : 02 June 2021

Published : 31 March 2022

Issue Date : February 2022

DOI : https://doi.org/10.1007/s11018-022-02017-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• calibration
• verification
• specified conditions
• calibration diagram
• correction function
• probability distribution
• convolution
• identification
• measurement errors
• inadequacy-related errors
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

https://www.nist.gov/pml/owm/laboratory-metrology/state-lab-program-resources

Office of Weights and Measures

State lab program resources.

The National Institute of Standards and Technology (NIST) Office of Weights and Measures manages the Laboratory Metrology Program that began with the New State Standards Program established by Congress in 1965 as part of its continuing support to the States. This program is designed to provide laboratory Recognition to NIST Handbook 143 , Program Handbook (including adoption of ISO/IEC 17025:2017), technical and administrative guidance and training, proficiency testing, and assistance to State legal metrology laboratories to ensure accurate and traceable measurements from NIST to the local jurisdictions. The program operates through continued partnership with the State laboratories to manage numerous measurement-related activities. Certificates of Measurement Traceability are issued upon evaluation of the laboratory's ability to make reliable metrological measurements. Resources on this list are primarily developed and distributed for State Weights and Measures programs and metrology staff. Other laboratory users might find these items of general or specific interest and are welcome to use and/or modify them for other applications. NVLAP Handbook 150-2 (2019) contains technical criteria for calibration laboratories in normative (required) Annexes that was previously published in both Handbook 143 and the NVLAP Technical Guides. The technical criteria are applicable for all Weights and Measures laboratories as explicitly referenced in Handbook 143.  

State Lab Resources

• 2022 State Laboratory Program Workload Survey PDF
• 2020 State Laboratory Program Workload Survey PDF
• State Calibration Scope/Summary and Laboratory Contact Information
• General Information - OWM [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Design and experiment of a soil-covering and -pressing device for planters, share and cite.

Lu, Q.; Zhao, J.; Liu, L.; Liu, Z.; Wang, C. Design and Experiment of a Soil-Covering and -Pressing Device for Planters. Agriculture 2024 , 14 , 1040. https://doi.org/10.3390/agriculture14071040

Lu Q, Zhao J, Liu L, Liu Z, Wang C. Design and Experiment of a Soil-Covering and -Pressing Device for Planters. Agriculture . 2024; 14(7):1040. https://doi.org/10.3390/agriculture14071040

Lu, Qi, Jinhui Zhao, Lijing Liu, Zhongjun Liu, and Chunlei Wang. 2024. "Design and Experiment of a Soil-Covering and -Pressing Device for Planters" Agriculture 14, no. 7: 1040. https://doi.org/10.3390/agriculture14071040

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Fellowships
• Business Models
• Mobile & Apps
• Audience & Social
• Aggregation & Discovery
• Reporting & Production
• Translations

ChatGPT is hallucinating fake links to its news partners’ biggest investigations

Over the past year, several major news media companies have signed on the dotted line with OpenAI, entering a content licensing partnership with the developer of ChatGPT. Most of those partnership announcements state that as part of the deals, ChatGPT will produce attributed summaries of each media company’s reporting and link to their publications’ websites.

Now, my reporting confirms that ChatGPT is hallucinating URLs for at least 10 other publications that are part of OpenAI’s ongoing licensing deals. These publications include The Associated Press, The Wall Street Journal, the Financial Times, The Times (UK), Le Monde, El País, The Atlantic, The Verge, Vox, and Politico.

In my testing, I repeatedly prompted ChatGPT to link out to these publications’ marquee articles, including Pulitzer Prize-winning stories and years-long investigations. These types of stories are editorial investments that can be both incredibly valuable to a brand’s reputation, and incredibly costly to produce.

All together, my tests show that ChatGPT is currently unable to reliably link out to even these most noteworthy stories by partner publications.

While the specific language differs, most partnered media companies have explicitly stated that ChatGPT will link out to their websites. “Queries that surface The Atlantic will include attribution and a link to read the full article on theatlantic.com,” reads The Atlantic’s licensing deal announcement from last month. “ChatGPT’s answers to user queries will include attribution and links to the full articles for transparency and further information,” reads a similar announcement by Berlin-based publisher Axel Springer from December 2023. OpenAI has also pitched news publishers “priority placement and ‘richer brand expression’ in chat conversations” and “more prominent link treatments” in ChatGPT, according to reporting earlier this year by Adweek on leaked OpenAI slide decks .

It is unclear how OpenAI can guarantee these attribution and citation features for its partners while the underlying ChatGPT product is regularly outputting broken links to those same websites.

OpenAI told me in a statement that it has not yet launched the citation features promised in its licensing contracts. “Together with our news publisher partners, we’re building an experience that blends conversational capabilities with their latest news content, ensuring proper attribution and linking to source material — an enhanced experience still in development and not yet available in ChatGPT,” said Kayla Wood, an OpenAI spokesperson. OpenAI declined to answer questions on the hallucinations I documented or explain how new features might address the problem of fake URLs.

From my testing across 10 publications, it appears that currently, ChatGPT is often doing what its predictive text generation does best: predicting the most likely version of the URL for a given story — rather than the correct one.

“The page you are trying to access does not exist”

To test ChatGPT’s ability to link out to its partner publications, I mostly prompted the chatbot to search the web for information on exclusive investigations by each respective outlet. For example, in 2019, the Financial Times broke news of a massive fraud scandal in the world of payment processing. Its investigation into Wirecard not only won awards, but prompted action by international regulatory bodies and contributed to the company’s swift decline, leading to its insolvency filing in 2020.

When I prompted ChatGPT to search the web for news articles on the Wirecard fraud scandal, ChatGPT answered that the FT broke the story in February 2019. But at first it only cited links to websites like Money Laundering Watch and Markets Business Insider , which had both aggregated the FT’s original reporting.

When I followed up and asked ChatGPT to share a link to the original story, it told me to read the story on the FT’s website at this URL: https://www.ft.com/content/44dcb5d2-2a29-11e9-b2e4-601dbf7d9eff The link led to a 404 error that stated, “The page you are trying to access does not exist.”

The Wirecard story wasn’t the only prestigious investigation to turn up in ChatGPT with a fake URL. In my tests, I documented hallucinated links to two Pulitzer Prize-winning stories, including The Wall Street Journal’s 2018 reporting on Donald Trump’s involvement in hush money payments made to Stormy Daniels and Karen McDougal during his presidential campaign. That reporting helped trigger a criminal investigation that recently culminated in a jury finding Trump guilty on 34 felony counts . Despite the investigation’s notoriety, the link offered by ChatGPT to the WSJ’s reporting once again landed on a 404 error. (In May, the WSJ’s parent company, Newscorp, signed a reported $250 million dollar licensing contract with OpenAI .)

“We are sharing insights with OpenAI to help create the best product experience for both users and publishers, where provenance and attribution are explicit, more accurate, and more intentional,” Rhonda Taylor, an FT spokesperson, told me in a statement, emphasizing that publisher citations are a work in progress. “The new experience is still under development and not yet live in ChatGPT, but the priority should be a better experience with high quality attribution, not speed of release.”

Several publications declined to share details about this updated “experience” or a general timeline for its release, but there have been early signs that ChatGPT is experimenting with how it cites its sources. In March, OpenAI started rolling out a new feature that makes links more prominent in ChatGPT, by including the name of a cited website in parentheses with a hyperlink to the specific story it is citing.

We're making links more prominent when ChatGPT browses the internet. This gives more context to its responses and makes it easier for users to discover content from publishers and creators. Browse is available in ChatGPT Plus, Team and Enterprise. pic.twitter.com/1ChlZvVMUy — OpenAI (@OpenAI) March 29, 2024

While hallucinated URLs did sometimes appear in these parentheses across my tests, these hyperlinks had a much higher rate of accuracy than the hyperlinks that appeared in other parts of ChatGPT’s responses. OpenAI declined to answer questions about how ChatGPT generates its hyperlinks or how the methodology for these two types of citations may differ.

Often in my tests ChatGPT would, on first pass, link out in parentheses to news outlets or blogs that don’t have partnership deals with OpenAI. For the most part, these articles aggregated major investigations by the likes of Le Monde , Politico and The Verge . With some exceptions, these URLs to aggregations were accurate. And to its credit, in almost all of my tests, ChatGPT was able to at least correctly name the outlet that broke a major news story, occasionally detailing the publication date and naming the author in its response.

It was when I asked ChatGPT to share a link to the first outlet that reported a given story, or to share a link to reporting that it had already correctly identified and summarized, that the URLs were most likely to break.

For example, I prompted ChatGPT to search the web for the first news article that exposed the use of popular copyrighted novels in Book3 — a database widely used by Silicon Valley AI developers to train LLMs. ChatGPT correctly answered that Alex Reisner broke that story in The Atlantic as part of an exclusive series .

The slug in the URL it provided included the correct vertical on the site, the correct publication month and year, and a plausible string of search-engine-optimized keywords: technology/archive/2023/09/books3-dataset-ai-copyright-infringement/675324 . (The broken link redirects to another technology story published in September 2023.)

Whoever chose the actual URL for The Atlantic’s Book3 investigation ended up making a similar, but marginally different, choice. This is the actual slug for one of the articles: technology/archive/2023/09/books3-database-generative-ai-training-copyright-infringement/675363/ . Unfortunately, close enough doesn’t cut it for URLs.

Examples like this seem to indicate that ChatGPT is, at times, outputting the most likely URL in its response — predicting what the slug for a story could be, without knowing what it actually is. Across my tests ChatGPT’s hallucinated links regularly followed the standard URL format of a given website, but got the specific words or numbers in that URL wrong. Sometimes when I repeated a question, ChatGPT would even output slight variations on the same fake URL, leading to a string of 404 errors.

“Hallucinations in LLMs are known issues. We’re raising to OpenAI any inaccuracies we encounter involving The Atlantic,” Anna Bross, a spokesperson for The Atlantic, told me in a statement. “We believe that participating with AI search in its early stages — and shaping it in a way that values, respects, and protects our work — could be an important way to help build our audience in the future.”

As I detailed in my previous reporting on the Business Insider Union’s letter to management, many journalists in newsrooms that have partnered with OpenAI have publicly expressed skepticism about ChatGPT’s potential as a search tool. In May, The Atlantic Union published its own open letter demanding more transparency from its employer about its contract with OpenAI. On Wednesday, The Atlantic published a story documenting similar problems with ChatGPT’s ability to link out to its own reporting.

“These hallucinations are deeply concerning, and point to why we raised questions about The Atlantic’s agreement with OpenAI,” said David A. Graham, a staff writer and member of the union’s editorial bargaining committee. “We need to know much more about what the agreement says, and the company must work with us to demand protections for the integrity of our journalism and The Atlantic’s legacy.”

Cite your sources

All of my testing used ChatGPT’s free and most widely accessible version, which only requires a basic login. Most of my testing was also conducted using GPT-4o (OpenAI’s latest multimodal model that offers real-time web browsing to generate ChatGPT’s responses). But I was also able to replicate the URL hallucinations using models without real-time web browsing.

For example, without using free GPT-4o credits, I asked ChatGPT to share a link to the first investigation into Hollywood director Bryan Singer’s sexual misconduct allegations.

ChatGPT correctly identified The Atlantic as the outlet behind the headline-making 2019 investigation , but wrongly stated the story ran in October 2014. Even though ChatGPT claimed it could not browse the web, it still provided a hallucinated link to the supposed 2014 article and suggested I read more there. That broken link redirected to a different October 2014 story on The Atlantic about the Nigerian militant group, Boko Haram .

The hallucinations were also not specific to English-language publications. ChatGPT hallucinated links to major national investigations in French by the publisher Le Monde, and stories in Spanish published by the outlet El País (owned by Prisa Media). Both international media companies entered content licensing deals with Open AI in March.

Several of the publications I tested also only announced their OpenAI licensing deals in the past two months. That includes The Verge and Vox ( owned by Vox Media ), The Wall Street Journal and The Times ( owned by NewsCorp ), and The Atlantic . But from my tests, it doesn’t appear that the length of time an OpenAI partnership has been ongoing has a strong bearing on whether or not ChatGPT, in its current form, will produce a hallucinated URL.

ChatGPT output fake URLs to Politico and Business Insider investigations. Both outlets are owned by Axel Springer, which signed its content licensing deal with OpenAI over six months ago for a reported “ tens of millions of euros .”

I also documented fake URLs to stories by the AP, which was the first major publisher to sign a licensing deal with OpenAI in July 2023. Nearly a year later, in our testing, ChatGPT was still unable to correctly link out to an two-year-long investigation on West African migrants that won the AP a Livingston Award for International Reporting earlier this month.

Overall, the stories I tested for were often groundbreaking investigations and articles that incited a wave of follow-up coverage, sometimes kicking off a years-long news cycle. For digital publishers, these types of stories are often expensive and core to building a brand’s reputation and audience. If a product used by more than 200 million people a month republishes the contents of this reporting without properly linking back to the source, the return on those editorial investments could take a hit.

My tests demonstrate that ChatGPT is hallucinating URLs frequently, and that the product is currently unable to reliably link out to the most noteworthy stories by its partners. That said, this was not a full audit of ChatGPT and I plan to follow up with more reporting on the technical factors that might be at play here. If these URL hallucinations are happening at scale, though, OpenAI would likely need to resolve the issue to follow through on its general pitch to news publishers. That includes both ChatGPT accurately citing publications it has licensing deals with and its commitment to becoming a dependable source of referral traffic to their websites.

Cite this article Hide citations

Deck, Andrew. "ChatGPT is hallucinating fake links to its news partners’ biggest investigations." Nieman Journalism Lab . Nieman Foundation for Journalism at Harvard, 27 Jun. 2024. Web. 29 Jun. 2024.

Deck, A. (2024, Jun. 27). ChatGPT is hallucinating fake links to its news partners’ biggest investigations. Nieman Journalism Lab . Retrieved June 29, 2024, from https://www.niemanlab.org/2024/06/chatgpt-is-hallucinating-fake-links-to-its-news-partners-biggest-investigations/

Deck, Andrew. "ChatGPT is hallucinating fake links to its news partners’ biggest investigations." Nieman Journalism Lab . Last modified June 27, 2024. Accessed June 29, 2024. https://www.niemanlab.org/2024/06/chatgpt-is-hallucinating-fake-links-to-its-news-partners-biggest-investigations/.

{{cite web     | url = https://www.niemanlab.org/2024/06/chatgpt-is-hallucinating-fake-links-to-its-news-partners-biggest-investigations/     | title = ChatGPT is hallucinating fake links to its news partners’ biggest investigations     | last = Deck     | first = Andrew     | work = [[Nieman Journalism Lab]]     | date = 27 June 2024     | accessdate = 29 June 2024     | ref = {{harvid|Deck|2024}} }}

To promote and elevate the standards of journalism

Covering thought leadership in journalism

Pushing to the future of journalism

Exploring the art and craft of story

The Nieman Journalism Lab is a collaborative attempt to figure out how quality journalism can survive and thrive in the Internet age.

It’s a project of the Nieman Foundation for Journalism at Harvard University .

• Subscribe to our email
• Follow us on Twitter
• Like us on Facebook
• Download our iPhone app
• Subscribe via RSS
• Tweet archive

We’ve been accidentally cooling the planet — and it’s about to stop

Humans’ fossil fuel burning has cooled the planet while warming it — presenting problems for the future.

Key takeaways

Summary is AI-generated, newsroom-reviewed.

• Fossil fuels cause both global warming and, to a lesser degree, cooling.
• As air pollution is reduced, global temperatures increase.
• Shipping regulations and China’s pollution clean-up have reduced sulfur and saved lives.
• Aerosols mask warming; exact cooling effect uncertain.
• Cleaning air without reducing CO2 and methane risks faster warming.

Did our AI help? Share your thoughts.

It is widely accepted that humans have been heating up the planet for over a century by burning coal, oil and gas. Earth has already warmed by almost 1.2 degrees Celsius (2.2 degrees Fahrenheit) since preindustrial times, and the planet is poised to race past the hoped-for limit of 1.5 degrees Celsius of warming.

But fewer people know that burning fossil fuels doesn’t just cause global warming — it also causes global cooling . It is one of the great ironies of climate change that air pollution , which has killed tens of millions, has also curbed some of the worst effects of a warming planet.

Tiny particles from the combustion of coal, oil and gas can reflect sunlight and spur the formation of clouds , shading the planet from the sun’s rays. Since the 1980s, those particles have offset between 40 and 80 percent of the warming caused by greenhouse gases.

And now, as society cleans up pollution, that cooling effect is waning. New regulations have cut the amount of sulfur aerosols from global shipping traffic across the oceans; China, fighting its own air pollution problem, has slashed sulfur pollution dramatically in the last decade.

The result is even warmer temperatures — but exactly how much warmer is still under debate. The answer will have lasting impacts on humanity’s ability to meet its climate goals.

“We’re starting from an area of deep, deep uncertainty,” said Zeke Hausfather, a climate scientist and research lead for the payments company Stripe. “It could be a full degree of cooling being masked.”

Most of the cooling from air pollution comes through sulfur aerosols, in two ways. The particles themselves are reflective, bouncing the sun’s rays away and shading the Earth. They also make existing clouds brighter and more mirror-like, thus cooling the Earth.

Coal and oil are about 1 to 2 percent sulfur — and when humans burn fossil fuels, that sulfur spills into the atmosphere. It is deadly: Sulfur dioxide has been linked to respiratory problems and other chronic diseases, and air pollution contributes to about 1 in 10 deaths worldwide.

Over the past few decades, countries have worked to phase out these pollutants, starting with the United States and the European Union, followed by China and India. China has cut its sulfur dioxide emissions by over 70 percent since 2005 by installing new technologies and scrubbers on fossil fuel plants. More recently, the International Maritime Organization instituted restrictions in 2020 on the amount of sulfur allowed in shipping fuels — one of the dirtiest fuels used in transportation. Shipping emissions of sulfur dioxide immediately dropped by about 80 percent. Mediterranean countries are planning a similar shipping regulation for 2025.

“There has been a pretty steep decline over the last 10 years,” said Duncan Watson-Parris, an assistant professor at the Scripps Institution of Oceanography at the University of California at San Diego.

These moves have saved lives — according to estimates, about 200,000 premature deaths have already been avoided in China, and the new shipping regulations could save about 50,000 lives per year. But they have also boosted global temperatures. Scientists estimate that the changes in aerosols from the new shipping rule alone could contribute between 0.05 and 0.2 degrees Celsius of warming over the next few decades.

Some researchers have suggested that the changes to ocean shipping regulations may have been a big contributor to last year’s record heat — and that aerosols may have been masking much more heat than previously thought. Satellite images have shown that cloud changes declined after sulfur emissions went down.

“The data from NASA satellites shows that in regions where this should be expected, there’s a very strong increase in absorbed solar radiation,” said Leon Simons, an independent researcher and a member of the Club of Rome of the Netherlands, pointing to shipping areas affected by the new rules. “And also in this period you see sea surface temperatures increasing in the same region.”

In one new paper , scientists at the University of Maryland argued that the decrease in aerosols could double the rate of warming in the 2020s, compared to the rate since 1980. But other researchers have critiqued their results.

Many experts believe the effect is likely to be modest — between 0.05 and 0.1 degrees Celsius. “I don’t think it’s possible to get better than a factor of two, in terms of how uncertain we are,” said Michael Diamond, a professor of meteorology and environmental science at Florida State University.

Some scientists see the shipping regulation as an analog to a way that researchers are exploring to halt global warming: purposefully brightening clouds using less polluting methods. In Alameda, Calif., researchers recently released sea salt aerosols into the atmosphere as a first step to study how the particles could brighten clouds and reflect sunlight. City officials later halted the project, despite reports showing that the experiment was safe.

But the real issue is still ahead. Currently, the U.N. Intergovernmental Panel on Climate Change estimates that aerosols are masking about 0.5 degrees Celsius of global warming. But that value could be as high as 1 degree or as low as 0.2 degrees — and the difference could be the difference between meeting the goals of the 2015 Paris agreement or not.

If aerosols have been masking cooling much more than expected, for example, the world could be poised to blow past its climate targets without realizing it.

Almost 200 of the world’s nations pledged in the Paris agreement to keep warming below 2 degrees Celsius (3.6 degrees Fahrenheit) compared with preindustrial levels. Scientists believe that many dangerous impacts, from the collapse of coral reefs to the melting of major ice sheets, will occur somewhere between 1.5 and 2 degrees Celsius.

“It’s not just a story of greenhouse gas emissions,” said Robert Wood, a professor of atmospheric science at the University of Washington. “Whether you clean up rapidly, or whether you just fumble along with the same aerosol emissions, could be the difference of whether you cross the 2-degree Celsius threshold or not.”

No scientists are advocating a halt to aerosol cleanup efforts — the death tolls from air pollution are simply too high. “There are really good reasons to want to be cleaning up air pollution,” Diamond said. “The public health benefits are really important.”

But researchers worry that cleaning up air pollution without halting fossil fuel use — as, for example, in China — could be a recipe for even greater and faster warming. “We need to make sure that we’re doing it at the same time as cleaning up methane and cleaning up CO2,” Diamond said. Cutting methane emissions, he noted, could help offset the effects of declining aerosols. Methane has a warming effect, but like aerosols, doesn’t remain in the atmosphere for very long.

Still, a lot of scientific questions remain — and until they are answered, the world won’t know exactly how much warming falling aerosols will unmask.

Harry Stevens contributed to this report.

More on climate change

Understanding our climate: Global warming is a real phenomenon , and weather disasters are undeniably linked to it . As temperatures rise, heat waves are more often sweeping the globe — and parts of the world are becoming too hot to survive .

What can be done? The Post is tracking a variety of climate solutions , as well as the Biden administration’s actions on environmental issues . It can feel overwhelming facing the impacts of climate change, but there are ways to cope with climate anxiety .

Inventive solutions: Some people have built off-the-grid homes from trash to stand up to a changing climate. As seas rise, others are exploring how to harness marine energy .

What about your role in climate change? Our climate coach Michael J. Coren is answering questions about environmental choices in our everyday lives. Submit yours here. You can also sign up for our Climate Coach newsletter .