design of experiments certification

Statistical Thinking for Industrial Problem Solving

A free online statistics course.

Back to Course Overview

Design of Experiments

Design of experiments (DOE) is a rigorous methodology that enables scientists and engineers to study the relationship between multiple input variables, or factors , on key output variables, or responses .

In this module, you will learn why designed experiments are better than trial and error and one-factor-at-a-time approaches to gain an understanding of cause and effect relationships and interactions between factors. You will be introduced to several types of designs such as factorial, response surface and custom designs. Finally, you will learn some DOE guidelines and best practices which will help you succeed with experimentation.

Estimated time to complete this module: 3 to 4 hours

Design of Experiments Overview (1:01)

Specific topics covered in this module include:

Introduction to doe.

• What is DOE?
• Conducting Ad Hoc and One-Factor-at-a-Time (OFAT) Experiments
• Why Use DOE?
• Terminology of DOE
• Types of Experimental Designs

Factorial Experiments

• Designing Factorial Experiments
• Analyzing a Replicated Full Factorial
• Analyzing an Unreplicated Full Factorial

Screening Experiments

• Screening for Important Effects
• A Look at Fractional Factorial Designs
• Custom Screening Designs

Response Surface Experiments

• Introduction to Response Surface Designs
• Analyzing Response Surface Experiments
• Creating Custom Response Surface Designs
• Sequential Experimentation

DOE Guidelines

• Introduction to DOE Guidelines
• Defining the Problem and the Objectives
• Identifying the Responses
• Identifying the Factors and Factor Levels
• Identifying Restrictions and Constraints
• Preparing to Conduct the Experiment

Design of Experiments (DoE) for Engineers PD530932

Topics: Quality, Safety & Maintenance Product development , Manufacturing processes , Design Engineering and Styling

How do you determine the root cause of a problem or identify which variable settings will make the product or process more "robust"? What if you need to gain a better understanding of a complicated system? Can you identify which variables most affect performance and obtain a well-correlated regression equation that explains how those selected system variables and their interactions affect performance?  

Design of Experiments (DOE) is an excellent, statistically based tool used to address and solve these questions in the quickest, least expensive, and most efficient means possible. It's a methodology that includes steps for identifying system variables worthy of study and the ideal experiment type to execute; for setting up an organized, efficient series of tests involving various combinations of selected variables; and for statistically analyzing the collected data to help obtain definitive answers to these problem-solving and optimization challenges. 

DOE is a methodology that includes steps for identifying system variables worthy of study and the ideal experiment type to execute; for setting up an organized, efficient series of tests involving various combinations of selected variables; and for statistically analyzing the collected data to help obtain definitive answers to these problem-solving and optimization challenges. 

This eLearning course utilizes a blend of text, videos, and hands-on activities to help you gain proficiency in executing designed experiments. It explains the pre-work required prior to DOE execution, how to select the appropriate designed experiment to run, and choosing the appropriate factors and their levels. You'll also learn how to execute the experimental tests ("runs") and analyze/interpret the results with the benefit of computer software tools, such as Minitab. 

You'll set up, run, and analyze simple-to-intermediate complexity Full Factorial, Partial Factorial, Taguchi/Robust, and Response Surface experiments both by hand and using computer software. You'll also receive an overview of Mixture experiments and information on how to install and configure a fully functional 30-day trial version of Minitab for completing practice activities and for personal evaluation. You'll gain the most value from this course by running experiments through various class exercises, with answers discussed after you've had the opportunity to execute the DOE on your own.

By participating in this on-demand course, you'll be able to:

• Determine when DOE is the correct tool to solve a given problem or issue
• Select the appropriate DOE experiment type (DOE goal) for a given application
• Set up simple Full Factorial DOEs by hand using cube plots
• Set up and analyze any Full Factorial DOE using Minitab®
• Identify appropriate Partial Factorial design(s) based on one's application
• Set up and analyze Partial Factorial DOEs, simple Robust Design (Taguchi) DOEs, and simple Response Surface DOEs using Minitab®
• Recognize the structured process steps recommended when executing a DOE project

Materials Provided

• 90 days of online  single-user  access  (from date of purchase)  to the seven and a half hour presentation
• Integrated knowledge checks to reinforce key concepts
• Online learning assessment (submit to SAE)
• Glossary of key terms
• Job aids (included in each module of published course)
• Instructions on how to access a 30-day trial of Minitab ®
• Video demonstrations of exercise solutions using Minitab ®
• Follow-up to your content questions
• 1.0 CEUs*/Certificate of Achievement (upon completion of all course content and a score of 70% or higher on the learning assessment)

*SAE International is authorized by  IACET  to offer CEUs for this course.

Is this On Demand Course for You?

This course will benefit engineers involved in problem-solving, such as product design or product formulation (e.g., fluid/material composition, prepared food recipes/preparation, etc.) and/or optimization; process design and/or optimization; quality improvement efforts, such as defect elimination, warranty avoidance or similar initiatives; test engineers who wish to maximize learning of system behavior with a minimum number of tests; and technicians, analysts, and managers who support engineers in the above efforts, so they may be effective participants in DOE activities.

Testimonial

"DOE expertise is a must have for engineers who deal with data all the time, whether it's in a simulation or test, or identifying the factors which have the most influence on the experiment." Raj Chandramohanan Sr. Project Engineer Borg Warner Inc.

"This course helped me to develop a good understanding of the DOE method and to apply it to real-world applications." Usman Asad Senior Research Associate University of Windsor

"Very insightful; it definitely helped me understand the different applications/uses of the DOE techniques." Alberto Aguilar Lead Engineer, EGR system PV&V John Deere Power Systems

For More Details

Email [email protected] , or call 1-877-606-7323 (U.S. and Canada) or 724-776-4970 (outside US and Canada).

"There are no specific course prerequisites; however, participants are expected to have some math background, including the ability to calculate elementary statistics parameters, such as an average and a range. Since the course includes demonstration and hands-on use of Minitab®, participants should have some familiarity with Windows-based personal computer applications. 

– If you’d like only an introduction or overview of the topic, this module can be purchased as a stand-alone course. – After completing the introductory module, you may purchase the remaining course modules as one package, without the need to repurchase the introductory module.  – If you'd like to take the complete course, this purchase option includes all the course modules in one package.

Design of Experiments Specialization

Program fee.

Well-designed experiments are a powerful tool for developing and validating cause and effect relationships between factors when evaluating and improving product and process performance. Deliberately changing the input variables to a system allows for observation and identification of the reasons for the change that may be observed in the output responses. Design of Experiments can identify important interactions that are usually overlooked when experimenters vary only one factor at a time (OFAT experimentation). Unfortunately, OFATS are still widely used in many experimental settings.

Design of Experiments can be used in a variety of experimental situations. This program is suitable for participants from a broad range of industries, including electronics and semiconductor, automotive, aerospace, chemical and process, pharmaceutical, medical device, and biotechnology. There are also many business and commercial applications of designed experiments, including marketing, market research, and e-commerce. Program participants will learn how to run effective and strong experiments using modern statistical software. 

Program Topics

We are proud to offer the Design of Experiments Specialization through the Coursera platform. The course is instructed by Dr. Doug Montgomery, a Regents Professor of industrial engineering and statistics in the Ira A. Fulton Schools of Engineering at ASU, and an expert in experimental design. Dr. Montgomery has taught academic courses on experimental design for over 40 years, and his Design of Experiments textbook, in its 10th edition and utilized in the specialization, is the most widely used textbook on the subject in the world. He has also led numerous engagements with Design of Experiments, teaching the course and consulting for more than 250 companies, including Motorola, Intel, Boeing and IBM. Drawing from these commercial experiences, Montgomery provides participants with an accurate understanding of modern approaches to using Design of Experiments.

The specialization is offered in a four-course format, with each course comprising three-to-four units and, in most courses, an applied project to demonstrate the tools and concepts learned. Accessible entirely online, the courses can be attempted at your own pace. We recommend completing one unit per week.

Live Fireside Chats

Unique to this specialization, Dr. Montgomery hosts monthly fireside chats using Zoom where he discusses different topics in the areas and application of Design of Experiments concepts. Drawing from his expertise and vast network, Dr. Montgomery is frequently joined by a special guest and expert in the topic area being discussed. Planned for the second Wednesday of every month, these chats are open to the public for viewing. During this time, viewers can ask questions to Dr. Montgomery and his guest related to Design of Experiments’ concepts, application, and situational experiences. The previous fireside chat recordings can be found here .

If you would like information on how to join the monthly live fireside chats, please contact us at [email protected] . 

Specialization Courses

Experimental Design Basics

Unit 1: Getting Started and Introduction to Design and Analysis of Experiments

Unit 2: Simple Comparative Experiments

Unit 3: Experiments with a Single Factor - The Analysis of Variance

Unit 4: Randomized Blocks, Latin Squares, and Related Designs

Factorial and Fractional Factorial Designs

Unit 1: Introduction to Factorial Design

Unit 2: The 2^k Factorial Design

Unit 3: Blocking and Confounding in the 2^k Factorial Design

Unit 4: Two-Level Fractional Factorial Designs

Response Surfaces, Mixtures, and Model Building

Unit 1: Additional Design and Analysis Topics for Factorial and Fractional Factorial  Designs

Unit 2: Regression Models

Unit 3: Response Surface Methods and Designs

Unit 4: Robust Parameter Design and Process Robustness Studies

Random Models, Nested and Split-Plot Designs

Unit 1: Experiments with Random Factors

Unit 2: Nested and Split-Plot Designs

Unit 3: Other Design and Analysis Topics

If you would like to take all four courses, we recommend taking them in the above order. Each subsequent course will build on materials from the previous.

Learning Outcomes

Learning outcomes are organized by course. By completing all four courses, participants will:

• Organize a step-by-step process for designing, conducting and analyzing that experiments will lead to successful results
• Collect, analyze, and interpret data to provide the knowledge required for project success
• Demonstrate effective use of a wide range of modern experimental tools that enable practitioners to customize their experiment to meet practice resource constraints
• Use the analysis of variance, ANOVA, to analyze data from single-factor experiments with several factor levels

Earning a Certificate

The Design of Experiments Specialization is offered 100% online and through the Coursera platform. Participants can complete any of the four courses to receive a certificate of completion, and can complete all four to receive the specialization, thus mastering experimental design.

Who Should Enroll

These courses are open to any that are interested in learning about experimental design tools. Any person working in modern industry can apply the tools acquired in these courses to their current and future positions.

Pre-requisites

We recommend working knowledge of a basic statistics course. The basic fundamentals will be covered in the Experimental Design Basics course.

Textbook and Software

The textbook used throughout the specialization is  Design and Analysis of Experiments, 10th Edition  by Dr. Douglas C. Montgomery. Students are recommended to purchase or rent the textbook, but are not required. The courses within the specialization also utilize JMP statistical software. Participants have access to a free trial in the courses.

Contact Information

For more information on professional programs or certifications contact:

Professional & Executive Education [email protected] (480) 727-4534

or fill out our "Request for Information" form at the bottom of the page. 

Request Information

Serving technical professionals globally for over 75 years. Learn more about us.

MIT Professional Education 700 Technology Square Building NE48-200 Cambridge, MA 02139 USA

Accessibility

Design and Analysis of Experiments

Download the Course Schedule

Explore innovative strategies for constructing and executing experiments—including factorial and fractional factorial designs—that can be applied across the physical, chemical, biological, medical, social, psychological, economic, engineering, and industrial sciences. Over the course of five days, you’ll enhance your ability to conduct cost-effective, efficient experiments, and analyze the data that they yield in order to derive maximal value for your organization.

Course Overview

THIS COURSE MAY BE TAKEN INDIVIDUALLY OR As part of THE  PROFESSIONAL CERTIFICATE PROGRAM IN BIOTECHNOLOGY & LIFE SCIENCES .

This program is planned for those interested in the design, conduct, and analysis of experiments in the physical, chemical, biological, medical, social, psychological, economic, engineering, or industrial sciences. The course will examine how to design experiments, carry them out, and analyze the data they yield. Various designs are discussed and their respective differences, advantages, and disadvantages are noted. In particular, factorial and fractional factorial designs are discussed in greater detail. These are designs in which two or more factors are varied simultaneously; the experimenter wishes to study not only the effect of each factor, but also how the effect of one factor changes as the levels of other factors change. The latter is generally referred to as an interaction effect among factors.

The fractional factorial design has been chosen for extra-detailed study in view of its considerable record of success over the last 30 years. It has been found to allow cost reduction, increase efficiency of experimentation, and often reveal the essential nature of a process. In addition, it is readily understood by those who are conducting the experiments, as well as those to whom the results are reported.

The program will be elementary in terms of mathematics. The course includes a review of the modest probability and statistics background necessary for conducting and analyzing scientific experimentation. With this background, we first discuss the logic of hypothesis testing and, in particular, the statistical techniques generally referred to as Analysis of Variance. A variety of software packages are illustrated, including Excel, SPSS, JMP, and other more specialized packages.

Throughout the program we emphasize applications, using real examples from the areas mentioned above, including such relatively new areas as experimentation in the social and economic sciences.

We discuss Taguchi methods and compare and contrast them with more traditional techniques. These methods, originating in Japan, have engendered significant interest in the United States.

All participants receive a copy of the text, Experimental Design: with applications in management, engineering and the sciences , Duxbury Press, 2002, co-authored by Paul D. Berger and Robert E. Maurer, in addition to extensive PowerPoint notes.

Participant Takeaways

• Describe how to design experiments, carry them out, and analyze the data they yield.
• Understand the process of designing an experiment including factorial and fractional factorial designs.
• Examine how a factorial design allows cost reduction, increases efficiency of experimentation, and reveals the essential nature of a process; and discuss its advantages to those who conduct the experiments as well as those to whom the results are reported.
• Investigate the logic of hypothesis testing, including analysis of variance and the detailed analysis of experimental data.
• Formulate understanding of the subject using real examples, including experimentation in the social and economic sciences.
• Introduce Taguchi methods, and compare and contrast them with more traditional techniques.
• Learn the technique of regression analysis, and how it compares and contrasts with other techniques studied in the course.
• Understand the role of response surface methodology and its basic underpinnings.
• Gain an understanding of how the analysis of experimental design data is carried out using the most common software packages.
• Be able to apply what you have learned immediately upon return to your company.

Who Should Attend

This course is appropriate for anyone interested in designing, conducting, and analyzing experiments in the biological, chemical, economic, engineering, industrial, medical, physical, psychological, or social sciences. Applicants need only have interest in experimentation. No previous training in probability and statistics is required, but any experience in these areas will be useful.

Program Outline

Class runs 9:00 am - 5:00 pm every day.

• Introduction to Experimental Design
• Hypothesis Testing
• ANOVA I, Assumptions, Software
• Multiple Comparison Testing
• ANOVA II, Interaction Effects
• Latin Squares and Graeco-Latin Squares
• 2K Designs (continued)
• Confounding/Blocking Designs
• Confounding/Blocking Designs (continued)
• 2k-p Fractional-Factorial Designs
• 2k-p Fractional-Factorial Designs (continued)
• Taguchi Designs
• Taguchi Designs (continued)
• Orthogonality and Orthogonal contrasts
• 3K Factorial Designs
• Regression Analysis I
• Regression Analysis II
• Regression Analysis III & Introduction to Response Surface Modeling
• Response Surface Modeling (continued), Literature Review, Course Summary

AMONG THE SUBJECTS TO BE DISCUSSED ARE:

• The logic of complete two-level factorial designs
• Detailed discussion of interaction among studied factors
• Large versus small experiments
• Simultaneous study of several factors versus study of one factor at a time
• Fractional experimental designs; construction and examples
• The application of hypothesis testing to analyzing experiments
• The important role of orthogonality in modern experimental design
• Single degree-of-freedom analysis; pinpointing sources of variability
• The trade-off between interaction and replication
• Response surface experimentation
• Yates' forward algorithm
• The reliability of estimates in factorial designs
• The usage of software in design and analysis of experiments
• Latin and Graeco-Latin squares as fractional designs; examples
• Designs with all studied factors at three levels
• The role of fractional designs in response surface experimentation
• Taguchi designs
• Incomplete study of many factors versus intensive study of a few factors
• Multivariate linear regression models
• The book and journal literature on experimental design

Testimonials

The type of content you will learn in this course, whether it's a foundational understanding of the subject, the hottest trends and developments in the field, or suggested practical applications for industry.

How the course is taught, from traditional classroom lectures and riveting discussions to group projects to engaging and interactive simulations and exercises with your peers.

What level of expertise and familiarity the material in this course assumes you have. The greater the amount of introductory material taught in the course, the less you will need to be familiar with when you attend.

Design of Experiments (DOE) Training (On-site or Virtual)

The objective of Design of Experiments Training is to provide participants with the analytical tools and methods necessary to:

• Plan and conduct experiments in an effective and efficient manner
• Identify and interpret significant factor effects and 2-factor interactions
• Develop predictive models to explain process/product behavior
• Check models for validity
• Apply very efficient fractional factorial designs in screening experiments
• Handle variable, proportion, and variance responses
• Avoid common misapplications of DOE in practice

Participants gain a solid understanding of important concepts and methods to develop predictive models that allow the optimization of product designs or manufacturing processes. Many practical examples are presented to illustrate the application of technical concepts. Participants also get a chance to apply their knowledge by designing an experiment, analyzing the results, and utilizing the model(s) to develop optimal solutions (in the 4-days DOE Training program). Minitab or other statistical software is utilized in the class.

Seminar Content (3 or 4 Days)

• What is DOE?
• Definitions
• Sequential Experimentation
• When to use DOE
• Common Pitfalls in DOE
• Planning an Experiment
• Implementing an Experiment
• Analyzing an Experiment
• Case Studies
• Design Matrix and Calculation Matrix
• Calculation of Main & Interaction Effects
• Interpreting Effects
• Using Center Points
• Variable & Attribute Responses
• Describing Insignificant Location Effects
• Determining which effects are statistically significant
• Analyzing Replicated and Non-replicated Designs
• Developing First Order Models
• Residuals /Model Validation
• Optimizing Responses
• Structure of the Designs
• Identifying an “Optimal” Fraction
• Confounding/Aliasing
• Analysis of Fractional Factorials
• Other Designs
• Sample Sizes for Proportion Response
• Identifying Significant Proportion Effects
• Handling Variance Responses
• Central Composite Designs
• Box-Behnken Designs
• Optimizing several characteristics simultaneously
• Planning the DOE(s)

Why is DOE Training Important?

Experimentation is frequently performed using trial and error approaches which are extremely inefficient and rarely lead to optimal solutions.  Furthermore, when it’s desired to understand the effect of multiple variables on an outcome (response), “one-factor-at-a-time” trials are often performed.  Not only is this approach inefficient, it inhibits the ability to understand and model how multiple variables interact to jointly affect a response.  Statistically based Design of Experiments provides a methodology for optimally developing process understanding via experimentation.

In this course, participants gain a solid understanding of important concepts and methods in statistically based experimentation.  Successful experiments allow the development of predictive models for the optimization of product designs or manufacturing processes.  Several practical examples and case studies are presented to illustrate the application of technical concepts.  This course will prepare you to design and conduct effective experiments.  You will also learn how to analyze the data from experiments to understand significant effects and develop predictive models utilized to optimize process behavior.

DOE has numerous applications, including:

• Fast and Efficient Problem Solving (root cause determination)
• Shortening R&D Efforts
• Optimizing Product Designs
• Optimizing Manufacturing Processes
• Developing Product or Process Specifications
• Improving Quality and/or Reliability
• Ensure designs are robust against uncontrollable sources of variation

Typical Attendees:

• Product and Process Engineers
• Design Engineers
• Quality Engineers
• Personnel involved in product development and validation
• Laboratory Personnel
• Manufacturing/Operations Personnel
• Process Improvement Personnel
• Six Sigma professionals

On-Site Training Courses

• Design of Experiments
• Advanced Design of Experiments
• DOE For Mixtures/Formulations
• SPC / Process Capability
• Advanced Statistical Process Control
• Reliability / Weibull Analysis
• Advanced Reliability Analysis
• Accelerated Life Testing
• Basic Statistics, Hypothesis Testing, & Regression
• Advanced Statistics, Hypothesis Testing, & Regression Topics
• Measurement System Assessment
• Introduction to Quality
• Problem Solving Methods

Related Resources

• Training Services
• Staff Members
• Our Partners
• Testimonials
• Client List
• White Papers

Design of Experiments Training

Design of experiments (doe) training.

Our Design of Experiments (DOE) training is a 3.5 or 4.5 day course which includes printed course materials and the software  Quantum XL . The course is targeted to the individual who has no experience in DOE and would like to learn to plan, setup, execute, analyze, and optimize using DOE.

Fundamental DOE Concepts Introduction to Regression Introduction to DOE Regression Analysis Two-Level Designs Three-Level Designs Design of Experiments Summary Monte Carlo Simulation (Optional) Robust Design (Optional)

Design of Experiments for Medical Device and Pharmaceutical Firms

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

Stat 503: design of experiments.

•   Overview
•   Materials
•   Assessment Plan
•   Prerequisites
•   Online Notes

The course will cover most of the material in the text, Chapters 1-15. The students will be required to use statistical computer software to complete many homework assignments and the project.

Course Topics

This graduate level course covers the following topics:

• Understanding basic design principles
• Working in simple comparative experimental contexts
• Working with single factors or one-way ANOVA in completely randomized experimental design contexts
• Implementing randomized blocks, Latin square designs and extensions of these
• Understanding factorial design contexts
• Working with two level, 2 k , designs
• Implementing confounding and blocking in 2 k  designs
• Working with 2-level fractional factorial designs
• Working with 3-level and mixed-level factorials and fractional factorial designs
• Simple linear regression models
• Understanding and implementing response surface methodologies
• Understanding robust parameter designs
• Working with random and mixed effects models
• Understanding and implementing nested and split-plot and strip-plot designs
• Using repeated measures designs, unbalanced AOV and ANCOVA
•   Designing Experiments

Course Author(s)

Dr. James L Rosenberger is the primary author of these course materials and has taught this course for many semesters in residence and online.

  This course uses Honorlock for proctored exams. For more information view O.3 What is a proctored exam? in the student orientation.

For most assignments the Minitab GLM or SAS Proc GLM and Proc Mixed commands will satisfy the computing requirements. Minitab Design Of Experiments (DOE) commands are also utilized extensively.

Students should already feel comfortable using SAS at a basic level, be a quick learner of software packages, or able to figure out how to do the required analyses in another package of their choice. Students who have no experience with programming or are anxious about being able to manipulate software code are strongly encouraged to take the one-credit courses in SAS in order to establish this foundation before taking courses that rely on this software.

SAS will be supported and sample programs will be supplied but you will be required to do some programing on your own. Due to different software applications, software versions and platforms there may be issues with running code. Students must be proactive in seeking advice and help from appropriate sources including documentation resources, other students, the teaching assistant, instructor or helpdesk.

Montgomery, D. C. (2020).  Design and Analysis of Experiments , 10th Edition, John Wiley & Sons. ISBN-13: 978-1119722106

Last updated: FA23

Assessment Plan

• 10 Homework assignments graded. 40% (10% penalty for late assignments)
• Experiment design and analysis project.10% (due last week of class)
• Two preliminary examinations. 15% each.
• Comprehensive final examination (proctored). 20%

PLEASE NOTE: This course may require you to take exams using certain proctoring software that uses your computer’s webcam or other technology to monitor and/or record your activity during exams. The proctoring software may be listening to you, monitoring your computer screen, viewing you and your surroundings, recording and storing any and all activity (including visual and audio recordings) during the proctoring process. By enrolling in this course, you consent to the use of the proctoring software selected by your instructor, including but not limited to any audio and/or visual monitoring which may be recorded.  Please contact your instructor with any questions . ( Read more... )

Prerequisites

STAT 501 (or STAT 462 ) and STAT 502

Excedify - Engineering Online Courses

Design of experiments (doe).

Learn design of experiments DOE from the basics to advanced concepts and how to apply it to improve any process, product or machine. Gain skills including how to plan and execute experiments, analyze the data using software tools like Minitab.

Course Outline

Section 1: introduction to doe and fundamentals.

What is DOE? And how can I use it to make better pizza!

Assignment 1

DOE Vs. OFAT.

Assignment II

Steps to apply DOE

Introduction to the P-Value

What is P-Value ?

Assignment 3

Summery for Section one

Section 2: Full factorial design

Welcome to Section 2

Defining the factors and the output.

Introduction to the DOE Designs and factor levels

Assignment 2

The concept of the full factorial design

Example (Replication and Center point method)

Assignment 4 (Bonus)

Summery for section 2

Section 3: Fractional factorial design

Th Fractional factorial design

The concept of the Fractional factorial design

Fractional factorial design (The Resolution)

The trade-off table

Practical example for the resolution

Section 4: Optimization methods (Response Surface method)

Optimization methods (Response Surface method) (The Section is in Progress)

Introduction to the response surface methodology (RSM)

Central Composite Design (CCD)

Practical example using CCD

Randomizing & Blocking

Section 5: DOE-Plan creation and Analyse (Using Minitab)

Case Study for Minitab

Download Minitab

Minitab (Part 1)

Minitab (Part 2)

Minitab (Part 3)

About this course

• Course Certificate
• 100% Online
• 30-Day Money Back Guarantee

Sample Lecture

Watch the sample lecture to get a first feeling of the course

Course Reviews

One of the best learning platforms, sabrina ahsan.

My experience with the DoE course is outstanding. The whole course is designed carefully with the best possible graphic description. It would take months if I would try to learn the DoE through other platforms. I am very much looking forward to en...

My experience with the DoE course is outstanding. The whole course is designed carefully with the best possible graphic description. It would take months if I would try to learn the DoE through other platforms. I am very much looking forward to enrolling in their other courses as well.

Nancy Maria

Extensive resources provided!

Pricing options

Choose the plan that suits you best. Enjoy 30-day money back guarantee on both plans

Regular price

3 Months payment plan

Looking to train your team?

Earn a certificate.

and prove your skills

Skills you will gain in the course

All the steps to apply a doe method.

How to choose the suitable factors for doe, and what the factor levels are.

The different doe designs (full factorial design, fractional factorial design, center point method and Central Composite Designs (CCD))

How to calculate the effect of the factors by using simple mathematical equations

Effect diagram, regression equation, alias structure.

What response surface method is, and how we use it to optimize a process.

How to carry out an experiment and what randomization and blocking are.

Some of the statistical basics, for example what P-Value is.

Using Program Minitab (Appling what we learned on a project from real life).

Mastering Data-Driven Decision Making with Design of Experiments (DOE)

Welcome to Excedify's self-paced online Design of Experiments (DOE) Training, a transformative learning experience that equips participants with the essential skills to drive data-driven decision making and process optimization. This comprehensive course offers the flexibility to learn at your own pace while benefiting from expert instructor support throughout your journey.

In today's fast-paced world, organizations rely on efficient experimentation and data analysis to enhance product quality and operational efficiency. The DOE methodology empowers professionals to make informed decisions through structured experimentation and statistical analysis.

Key Learning Objectives: Our engaging online DOE Training is designed to help you achieve the following key objectives:

Fundamental Principles: Gain a deep understanding of DOE fundamentals, including the principles of experimental design, factors, levels, and responses.

Planning Effective Experiments: Learn how to strategically plan experiments to optimize resources and meet specific research or process improvement goals.

Data Collection and Analysis: Acquire hands-on experience in data collection methods and statistical analysis using popular software tools, all within the online platform.

Interpretation and Inference: Develop proficiency in interpreting experimental results, identifying influential factors, and drawing meaningful conclusions.

Advanced DOE Techniques: Explore advanced DOE methodologies like factorial designs, response surface methodologies, and mixture experiments for complex optimization challenges.

Real-World Applications: Apply DOE concepts to real-world case studies and practical scenarios, with the flexibility to access course material and exercises whenever convenient for you.

Who Should Attend: Our self-paced online DOE Training welcomes a diverse audience, including engineers, researchers, R&D professionals, data analysts, quality assurance experts, and anyone engaged in process improvement initiatives. The course accommodates your busy schedule, allowing you to learn and progress at your own speed.

Course Features:

• 100% Online: Study at your own convenience and pace, accessing course materials anytime, anywhere.
• Instructor Support: Benefit from expert guidance and support from experienced instructors throughout your learning journey.
• Interactive Exercises: Engage in interactive exercises and simulations to reinforce your understanding of DOE concepts.
• Certificate of Completion: Receive a prestigious certificate upon successfully completing the course, showcasing your expertise in Design of Experiments.

Embark on a Journey of Excellence: At Excedify, we believe in empowering professionals like you with knowledge and skills that lead to organizational success. Enroll in our self-paced online Design of Experiments (DOE) Training and gain the confidence to drive data-driven decision making, optimize processes, and achieve your professional goals.

To enroll or inquire further, please contact us at [email protected] . Our team is eager to support your learning journey and ensure a fulfilling experience with our DOE Training.

Empower yourself and your organization with data-driven excellence. Enroll in the Design of Experiments (DOE) Training today!

Instructor(s)

Paul Bradley

Presentation and delivery.

Senior Instructor - Project Management and DoE

we offer a 30-day money-back guarantee period in which you can view and engage with all the course material. If you did not love the course for any reason, we will return 100% of the paid price. If you issue the refund after you completed the course, we will refund 50% of the paid price and the course certificate will be revoked.

Yes, you can pay using one of the following methods: just email us at [email protected] stating which course you would like to buy. PAYMENT METHODS: Apple Pay; Google Pay; Alipay; WeChat Pay; Bancontact; EPS; giropay; iDEAL; Przelewy24; Sofort; Klarna; Bank debits; SEPA Direct Debit;

Yes, you can buy a bulk license with one custom order. Just email us at [email protected] with the required number of users and the course of interest. We will take care of the bulk enrollment and account setup. We offer company communities and progress reporting for our B2B customers.

Additional products

Six Sigma Black Belt Training

Our Six Sigma Black Belt Training is an advanced program that equips participants with the knowledge and skills to lead complex process improvement projects. Learn advanced statistical analysis, problem-solving techniques, and project management.

Agile Project Management

Join our Agile Project Management Fundamentals course to master the principles and methodologies of Agile project management. Discover how to plan, execute, and adapt projects using Agile techniques, promoting collaboration and achieving success.

MSA Measurement Systems Analysis

This Measurement Systems Analysis (MSA) course equips learners with the knowledge and skills needed to evaluate, improve, and validate measurement systems in a variety of settings.

Engineering Design Specialization Program

This professional design engineer specialization program is designed to provide students with the knowledge and skills needed to excel in the field of engineering design.

• Software Stat-Ease 360® Design-Expert® Academic Licensing Tutorials Registration Testimonials Compliance Statements International Distributors Installation FAQ Downloads & Updates
• Learn DOE Quick Start Resources Public Training On-Site Training Webinars Stat-Ease Academy (eLearning) Student Testimonials
• Publications Books Case Studies/White Papers The Stat-Teaser Newsletter DOE FAQ Alert Blog Newsroom
• Events Event List DOE Meetings Webinars

Design of Experiments Training

Targeted topics provide faster insights, leading to immediate results! Our interactive learning experience promotes real understanding of DOE methods and tools.

Stat-Ease 360 Software

SE360 augments Design-Expert software with integration with Python scripting for data analysis, new Custom Graphs, and Gaussian process models for zero-error and noisy responses.

Hosted Network Licensing Available

Our Hosted Network License allows you to run the software on multiple devices without requiring an on-premises license server. Ask us for a quote!

Design-Expert Software

Best in class design of experiments software makes R&D easy with an intuitive interface and amazing graphics. Whether you are new to DOE or a seasoned pro, you'll make breakthrough improvements with Design‑Expert software.

New-User Intro to Stat-Ease® 360 Software

A complete introduction to Stat-Ease DOE software, this free webinar provides new users with the tools needed to start planning your DOE.

A better product. An improved process. Stat-Ease, armed with an expertise in design of experiments, has been helping people make breakthroughs since 1985. Find out how our industry-leading software and multi-modal training can help you improve your bottom line.

Make the most from every experiment sm

Stat-Ease Software

Powerful tools for experimenters.

Make breakthrough improvements with Stat-Ease 360 or Design-Expert® software. Our software makes it incredibly simple to apply powerful multifactor testing tools.

Whether you are new to design of experiments or a pro, our straightforward, easy-to-use interface sits on top of a powerful statistical engine. You will not only screen for vital factors, but also locate ideal process settings for top performance & discover optimal product formulations.

The latest releases provide new Custom Graphs, a multi-response Analysis Summary view, and the ability to Import/Export data to Excel. View the software brochure here .

For information on the latest version, view our change log .

Hands-on DOE Training

Learning at your convenience.

Learn design of experiments (DOE) your way! Stat-Ease offers premium training options for Design-Expert and Stat-Ease 360 alongside general DOE education.

Take advantage of our online eLearning modules designed for self-directed DOE learning, then work through our step-by-step software tutorials . Learn more by registering for a free one-hour webinar .

For comprehensive DOE education, enroll in our highly interactive instructor-led workshops for either process DOE or mixture formulations. Experienced experts lead our hands-on, case-study based courses. Whatever your educational style, we offer training that meets your needs.

Version 23.1.5 Now Available

All subscribers of Stat-Ease software or users with an active Annual Support & Maintenance plan are eligible to install and run the latest versions of Stat-Ease 360 or Design-Expert as they are released. Check our changelog for the details on what's new, and download the latest version from the Licenses tab in your account .

Full 2024 Workshop Schedule Available

Our public workshop schedule for the rest of 2024 is live! Register now for either our Mixture Design for Optimal Formulations or our Modern DOE for Process Optimization course. If you can't make the scheduled time, or if you have more than 5 people to train, contact us for options . We look forward to helping you save time and money with design of experiments!

New 3-part series on Robust Design

Stat-Ease expert instructor Richard Williams has written a hefty series of blog posts on robust design. All three parts, plus a bonus follow-up from Stat-Ease Engineering Consultant Mark Anderson, are now live! Learn how to demonstrate robustness against external noise factors, internal process variation, and combinations of both.

Follow-Up by Mark Anderson

• Testimonials

• University Partners
• CORPORATE PARTNERS
• SUGGEST A COURSE
• TEACH A COURSE

Design of Experiments (DOE) Online Course & Certificate

• Course Description
• Course Outcome
• Prerequisites
• Course Delivery

Design of experiments (DOE) is a systematic, rigorous approach to engineering problem-solving that applies principles and techniques at the data collection stage so as to ensure the generation of valid, defensible, and supportable engineering conclusions. In addition, all of this is carried out under the constraint of a minimal expenditure of engineering runs, time, and money.

Design of experiments is a basic course in designing experiments and analyzing the resulting data. It is intended for engineers, scientists, and business professionals. The course deals with the types of experiments that are frequently conducted in industrial settings.

Applications from various fields of engineering (including chemical, mechanical, electrical, materials science, industrial, etc.) will be illustrated throughout the course. Computer software packages (Minitab) to implement the methods presented will be illustrated extensively, and you will have opportunities to use it for homework assignments and the term project.

After completing Design of Experiment, students will be equipped with a basic understanding of the following:

• how to plan, design and conduct experiments efficiently and effectively
• how to analyze the resulting data to obtain objective conclusions
• how to use the principles in all phases of engineering work, including new product design and development, process development, and manufacturing process improvement.

This course is designed for scientists, engineers, or business professionals who desire to learn statistical and design experiments. Completion of Statistics: An Introduction for Beginners recommended.

 Students will be assigned 5 homework assignments, and 1 final exam. 

Introduction

Preparation

Components of Experimental Design

Purpose of Experimentation

Design Guidelines

Design Process

One Factor Experiments

Multi-factor Experiments

Taguchi Methods

Plackett Burman experiment

Computer software (Minitab) examples

 This course is Instructor-led and delivered through our award-winning online Learning Management System. 

Course Content

Subscribe to our newsletter to stay up to date.

Username or Email Address

Remember Me

What's the Difference?

Objective Experiment Strategies covers all of the basic concepts of Design of Experiments.

Objective Experiment Strategies for Biotech covers all of the basic concepts of Design of Experiments with exercises focused on Biotech.

Objective Experiment Strategies for Chemistry covers all of the basic concepts of Design of Experiments plus the more advanced topic of mixture designs. It focuses on chemical experiments.

You Have a Choice of Design of Experiments Workshops

• Objective Experiment Strategies (3 days)
• Objective Experiment Strategies for Biotech (3 days)
• Objective Experiment Strategies for Chemistry (4 days)
• Customized Design of Experiments Workshop

How to Choose the Right Workshop for You

Objective Experiment Strategies is right for you if:

• You want a complete introduction to Design of Experiments.
• You don't need to work with discrete factors or non-Normal data.
• You don't perform chemistry experiments involving mixtures.

Objective Experiment Strategies for Biotech is right for you if:

• You want to focus on applications specific to Biotech.

Objective Experiment Strategies for Chemistry is right for you if:

• You are a Chemist or Chemical Engineer.
• You want a thorough introduction to Design of Experiments including mixture designs.

A customized course is right for you if:

• You work with discrete (categorical) factors.
• You work with non-Normal data.
• You work with mixtures, but aren't a Chemist.
• You have some other special requirement(s).

Would You Like Help Deciding?

Please contact us at

(888) 764-3958

and we'll be happy to help you.

Certification is available.

Due to scheduled OIT maintenance, our website is operating at a reduced capacity. New user account creation, online course access, course registration, and payments will be temporarily unavailable from 4:00pm until 11:00pm today(Friday, June 28).

• Defense Technologies
• Digital Media
• Engineering
• K-12 Programs
• Manufacturing
• Mathematics
• Occupational Safety & Health
• Supply Chain & Logistics
• Graduate Certificates
• Professional Certificates
• Savannah Campus
• Workplace Learning & Professional Development
• Corporate Education
• Train at Your Location
• Georgia Tech Summer
• Military Programs
• ESL (English as a Second Language)
• Georgia Film Academy
• Online Courses
• Massive Open Online Courses (MOOCs)
• Global Learning Center
• Savannah Facilities

Design of Experiments (DOE) I: Introduction to DOE

• Course Content
• Requirements & Materials

COURSE ID: DEF 5003P

Contact for course-related questions

Instructors

Course Description

A properly designed experiment should be efficient, informative, and directional. Sadly, technical professionals are almost never taught the rigorous techniques of experimentation that allow them to make informed, statistically meaningful decisions. This course introduces students to the long-lost technique of factorial experimentation where, upon course completion, a student will be almost certain to exclaim, “why was I never taught this in school?!”

In this course, you will learn how to efficiently derive a mathematical representation of a complex system that can be used to inform you about system behavior, predict outcomes with statistical confidence, and direct innovation in a meaningful way.    

In addition this course will provide hands-on experience through Statapults, simulations, and case studies. A brief introduction into specialized DOE software will also be used as a means to gain experience into the tools DOE practitioners use to build and analyze their experiments.

Due to minimum enrollment requirements to hold a course, we ask that you register as early as possible. View our terms and conditions for further information. If you have specific questions or requirements, please contact the course administrator. 

INTRODUCTION TO DESIGN OF EXPERIMENTS

• Introduction to DOE process and setup
• Learn how to understand the statistical output metrics
• Review of statistics, confidence, and statistical power
• Understand sample sizing to test enough for the problem at hand

BASIC DOE EXAMPLES

• Linear full factorial designs
• Quadratic full factorial designs
• Mixed level DOE designs
• Recovering from a missed quadratic effect

FINDING INTERACTIONS

• The importance of interaction effects
• Why fractional factorial designs are used
• How to determine the aliasing patterns and what they mean
• Detecting possible problems with confounding
• Recovering from a missed interaction effect

OTHER COMMON DESIGNS

• Screening designs
• Optimal designs
• Incrementally increasing the complexity of designs

HANDS-ON DOE SOLUTIONS

• Using simple Microsoft Excel add-in software for statistical tests and DOE solutions
• Using DOE-specific software for statistical power calculations and trade-offs
• Modeling distances shot from Statapults and deriving linear full factorial, augmented quadratic , and full factorial quadratic models from the shot data and using DOE software

Prerequisites

Recommended

• Working knowledge of probability and statistics
• Textbook and course notes
• Single-output Design of Experiments software package
• Laptop computer

Session Details

Special Discounts: Georgia Tech Research Institute (GTRI) employees are eligible to receive a discount.  If you are a GTRI employee, please go to the  Organizational Development website and look for the coupon code under GT Professional Development . Review coupon instructions for more information.

Upcoming Sessions

Jul 16, 2024 - jul 18, 2024 register by jul 12, 2024 atlanta, ga $1,400, items to purchase, additional steps required.

For those who plan to have their registration fees paid for by their company, we offer several secure options. These include a convenient online tool that streamlines the payment process for both groups and individuals.

However, there are a few items to prepare prior to the registration process. Please visit this page for complete details.

Aug 20, 2024 - Aug 22, 2024 Register By Aug 15, 2024 Las Vegas, NV $1,400

Sep 10, 2024 - sep 12, 2024 register by sep 5, 2024 lake buena vista, fl $1,400, previous sessions, who should attend.

This course is designed for any technical professional who is frustrated by the length and monotonous nature of experimental testing.  This includes, but is certainly not limited to, engineers who want to shorten their testing duration of a complex system, technicians who want to optimize machine efficiency, data analysts who want to understand complex relationships driving KPIs, industrial designers interested in reducing assembly time, or a lean practitioners looking to reduce system variance.  Any technical professional interested in characterizing complex systems can benefit from this course.

What You Will Learn

• How factorial experimentation is superior to other common, more rudimentary methods of experimentation
• How to answer the question:   “How much testing is enough?”
• How to answer the objection:   “This is a lot of testing.   How can you justify the cost?”
• The method of sequential experimentation and how it is critical to being efficient with resources
• How to brainstorm with teams in a way that answers the question, “what should we test and why?”
• How to use system characterization questions to predict behavior
• Semi-advanced techniques including  fractional factorial experiments, screening designs, and optimal designs, and sequential experimentation
• Solution of DOE problems using examples from Statapults, computer games, and simulations
• How to design an experiment properly with  modern software tools

How You Will Benefit

• Recognize how to format the problem or evaluation to take advantage of the DOE process and solution.
• Characterize the system under test or the system to be analyzed.
• Derive equations that explain the behavior of the response based on the factors and the behavior of the variability of the response based on the factors.
• Examine cause and effect, and verify controllable inputs and accurate, repeatable measurement systems.
• Understand how to generate orthogonal designs and their benefits.
• Efficiently design and conduct experimental studies for comparative evaluation, input-output characterization, output variance control, input sensitivity, and process control/optimization.
• Validate transfer function results or recover from lack of confirmation.

Grow Your Professional Network

Taught by experts in the field.

The course schedule was well-structured with a mix of lectures, class discussions, and hands-on exercises led by knowledgeable and engaging instructors.

Related Programs

Access (ADA)

The Georgia Tech Global Learning Center and Georgia Tech-Savannah campus is compliant under the Americans with Disabilities Act. Any individual who requires accommodation for participation in any course offered by GTPE should contact us  prior to the start of the course.

Courses that are part of certificate programs include a required assessment. Passing criteria is determined by the instructor and is provided to learners at the start of the course.

CEUs are awarded to participants who attend a minimum of 80% of the scheduled class time.

• Citizenship

Georgia Tech’s Office of Research Security and Compliance requires citizenship information be maintained for those participating in most GTPE courses. Citizenship information is obtained directly from the learner at the time of registration and is maintained in the Georgia Tech Student System.

Code of Conduct

Learners enrolled in any of Georgia Tech Professional Education's programs are considered members of the Georgia Tech community and are expected to comply with all policies and procedures put forth by the Institute, including the  Student Code of Conduct  and Academic Honor Code .

Course Changes and Cancellations

Please refer to our  Terms and Conditions  for complete details on the policies for course changes and cancellations.

Data Collection and Storage

Participants in GTPE courses are required to complete an online profile that meets the requirements of Georgia Tech Research Security. Information collected is maintained in the Georgia Tech Student System. The following data elements are considered directory information and are collected from each participant as part of the registration and profile setup process:

• Full legal name
• Email address
• Shipping address
• Company name

This data is not published in Georgia Tech’s online directory system and therefore is not currently available to the general public. Learner information is used only as described in our Privacy Policy . GTPE data is not sold or provided to external entities.

Sensitive Data The following data elements, if in the Georgia Tech Student Systems, are considered sensitive information and are only available to Georgia Tech employees with a business need-to-know:

• Georgia Tech ID
• Date of birth
• Religious preferences
• Social security numbers
• Registration information
• Class schedules
• Attendance records
• Academic history

At any time, you can remove your consent to marketing emails as well as request to delete your personal data. Visit our GTPE EU GDPR page for more information.

Inclement Weather

Classes and events being held at the Georgia Tech Global Learning Center in Atlanta or Georgia Tech-Savannah campus may be impacted by closures or delays due to inclement weather.

The Georgia Tech Global Learning Center will follow the guidelines of Georgia Tech main campus in Atlanta. Students, guests, and instructors should check the  Georgia Tech homepage  for information regarding university closings or delayed openings due to inclement weather. Please be advised that if campus is closed for any reasons, all classroom courses are also canceled.

Students, guests, and instructors attending classes and events at Georgia Tech-Savannah should check the  Georgia Tech-Savannah homepage  for information regarding closings or delayed openings due to inclement weather.

Program Completion

GTPE certificates of program completion consist of a prescribed number of required and elective courses offered and completed at Georgia Tech within a consecutive six-year period. Exceptions, such as requests for substitutions or credit for prior education, can be requested through the petition form . Exceptions cannot be guaranteed.

Please refer to our  Terms and Conditions  for complete details on the policy for refunds.

Smoking & Tobacco

Georgia Tech is a tobacco-free and smoke-free campus. The use of cigarettes, cigars, pipes, all forms of smokeless tobacco, and any other smoking devices that use tobacco are strictly prohibited. There are no designated smoking areas on campus.

Special Discounts

Courses that are eligible for special discounts will be noted accordingly on the course page. Only one coupon code can be entered during the checkout process and cannot be redeemed after checkout is complete. If you have already registered and forgot to use your coupon code, you can request an eligible refund . GTPE will cancel any transaction where a coupon was misused or ineligible. If you are unsure if you can use your coupon code, please check with the course administrator.

GTPE does not have a program for senior citizens. However, Georgia Tech offers a 62 or Older Program for Georgia residents who are 62 or older and are interested in taking for credit courses. This program does not pay for noncredit professional education courses. Visit the Georgia Tech Undergraduate Admissions page for more information on the undergraduate program and  Georgia Tech Graduate Admissions  page for more information on the graduate program.

Group Registrations

How do i register my group.

There is no special process or form to register your group. All interested learners must create and manage their own individual profiles, accounts, and registrations.  

• Complete a GTPE profile .
• Shop for a course .
• Add the course(s) to the cart.*
• Apply a group discount code (if applicable).
• Provide an accepted payment method to complete the order (credit card, third party credit card holder, or one accepted payment document).  

*Carts will remain active for 14 days, but seats are not held until the transaction is complete.

How do I apply for a group discount?

Courses that offer group discounts will display the discount code on the course page. Your employees will use the code during the registration process and cart totals will adjust accordingly. Group discounts can only be used if three or more employees from the company attend the same course and only one coupon code can be use per shopping cart.

If you have already registered and forgot to use your coupon code, you can request an eligible refund .

What are the accepted payment documents if I am unable to pay by credit card?

Accepted payment documents must be uploaded during the registration process. They include:

• A company purchase order (PO or SF182)
• A letter of authorization on company letterhead
• A corporate education application/voucher

What are the requirements for payment documents?

• Name of company and physical address
• Name of employee(s) approved for training
• Document number (SF-182 documents: Section C, Box 4)
• Billing address (SF-182 documents: Section C, Box 6)
• Course title and course dates
• Maximum disbursement amount (billing amount)
• Expiration date (if applicable)
• Authorized signature(s)
• Payment terms less than or equal to net 30

The employee can print of a copy of their shopping cart to submit if required for payment documents. The cart will remain active for 14 days, but the seat will not held until registration and payment is complete.

Registrations cannot be processed without payment. If your employee is concerned about losing a seat in a class because of internal company processes, we suggest that they go ahead and register and pay with a personal or corporate credit card and seek reimbursement.

Who can I contact for assistance?

If you need assistance with your group registration or have questions on how to start the process, please feel free to contact us at 404-385-3501 or [email protected] .

Individual Registrations

Where are your courses held.

Most GTPE classroom courses are held at the Georgia Tech Global Learning Center (GLC). Any courses that are held elsewhere will be clearly marked on the course page. Get information on parking, directions, and transportation to the GLC.

Do you provide overnight guest rooms?

We do not provide overnight rooms. However, accommodations can be made at the Georgia Tech Hotel and Conference Center, adjacent to us. Additional hotels can be found within walking distance. Get more information on accommodations .

What if I need to transfer to another course?

Learners may transfer to another course of equal or greater cost if notification is made at least 10 business days prior to the original course start date. The course to which one transfers must already be scheduled.

When should I register for a course?

We recommend you register for courses as early as possible. Session details will indicate when there is less than five reamining seats in a particular session.

How can I make updates to my contact information?

Updates to your company, address, email, phone, and passwords can be made directly on the GTPE website. Name changes and citizenship changes must be submitted to the GTPE Registrar’s Office.

How can I register for a course?

• Apply a special discount code (if applicable).

Do you accept walk-in registrations?

Walk-in registrations are accepted based on space availability but are not guaranteed for any courses.

Do you offer special discounts?

If available, discounts will display on the course page or will be automatically applied during the purchase process. Only one coupon code should be entered during the checkout process and will be validated by the system if applicable to items in your cart. If you have already registered and forgot to use your coupon code, you can request an eligible refund .

GTPE does not have a discount program for senior citizens. However, Georgia Tech offers a 62 or Older Program for Georgia residents who are 62 or older and are interested in taking for credit courses. This program does not pay for noncredit professional education courses. Visit the  Georgia Tech Undergraduate Admissions  page for more information on the undergraduate program and the  Georgia Tech Graduate Admissions  page for more information on the graduate program.

What professional education programs are eligible for veteran education benefits?

The following GTPE programs are eligible for veteran education benefits:

• Construction Safety and Health Certificate Program (Atlanta campus courses only)
• Project Management Certificate Program (Atlanta campus courses only)
• Safety and Health Management Certificate Program (Atlanta campus courses only)

View the GTPE veteran’s GI Bill benefits checklist for more information.

Do you have a program for senior citizens?

GTPE does not have a program for senior citizens. However, Georgia Tech offers a 62 or Older Program for Georgia residents who are 62 or older and are interested in taking for credit courses. This program does not pay for noncredit professional education courses. Visit the Georgia Tech Undergraduate Admissions page for more information on the undergraduate program and the Georgia Tech Graduate Admissions page for more information on the graduate program.

What happens if my course is cancelled?

In the event of a cancellation, we will provide you with a full refund or transfer to an equivalent course.

Do I need a student visa to take a course?

Short courses (1-5 days) and conferences do not require a student visa. A B-2 Tourist Visa, along with a copy of your registration confirmation email and a copy of your completed web registration order page, should suffice.

If participation in a course is employment related, with immediate departure from the U.S., then a B-1 Temporary Business Visa will be required.

We encourage you to contact your U.S. Consulate or Embassy to determine visa eligibility. Full refunds will be provided to participants who are unable to obtain an entry visa and contact our office prior to the start of the course.

English as a Second Language students should contact the Language Institute for admission and visa requirements.

Do you provide letters of invitations or immigration documents for student visas?

We do not issue letters of invitation and cannot provide immigration documents for the issuance of a student visa. Full refunds will be provided to participants who are unable to obtain an entry visa and contact our office prior to the start of the course.

What payment methods do you accept?

Full payment is due at time of registration. Accepted payment methods include:

• Credit cards
• Purchase orders (company and government)
• International wire payments *
• Georgia Tech Workday Number (for Georgia Tech employees only)
• Private loans *
• GI Bill benefits * (Eligible Atlanta campus courses only. Off-campus and online courses are not eligible for VA Benefits.)
• Company checks*

* Requires document upload or transaction verification during the checkout process.

What information is needed for a purchase order?

Purchase order documents must include the following:

• Name of company’s financial contact and/or ap (accounts payable) email address

Please do not include social security numbers on purchase order documents.

How do I pay with a company check?

• Make your check payable to “Georgia Institute of Technology” and include the order number and participant name on the face of the check.
• Choose “Company Purchase Order” as the payment method at checkout and upload a copy of your check to your order.

GTPE Accounting Georgia Institute of Technology Global Learning Center 84 5th St. NW Atlanta, GA 30308-1031

When is a payment due for a course?

Full payment is due at the time of registration.

How do I make a payment?

General Public Payment is due at the time of purchase. Invoice payments must adhere to the Board of Regent’s business terms of net 30.

Georgia Tech Employees PeopleSoft payments are processed at the time of registration. Georgia Tech employees cannot use PCards for GTPE registration charges.

Are there additional fees for books, supplies, or materials?

Additional fees vary by course. Be sure to review the Requirements & Materials tab on the course page for more information.

My company has offered to pay for this course. Can you invoice them directly?

Yes. Here are the steps to receive an invoice:

• Add the course(s) to the cart.
• Print your cart and submit to your employer as the cost estimate.
• Receive a copy of your company’s Purchase Order or payment approval document for GTPE to invoice against.
• Return to your cart, proceed through checkout and upload our company PO in the final payment step.
• The GTPE Business Office will generate an invoice 30 days prior to the start of the course at which point you are no longer eligible to withdraw with refund. Your company must:
• Abide by the Georgia Tech and Board of Regent’s business terms of net 30.
• Pay the full balance of a Georgia Tech invoice (there are no discounts for payments made early or on time).
• Pay the invoice if the employee fails to withdraw during the refund period and does not attend the course.

What is your policy for refunding a credit card payment?

Credit card refunds are processed to the original credit card. The credit card issuer is responsible for refund credit balances to the cardholder.

Do you offer payment plans?

We do not offer payment plans for any of our services, conferences, or courses. Payment must be made in full at time of purchase.

Will participants be issued a 1098-T tax form for courses taken at GTPE?

GTPE cannot issue 1098-T tax forms. If you have a payment history need for tax purposes, we are happy to provide you with receipts of payment. Please submit your requires to [email protected] . Be sure to include your full legal name and Georgia Tech ID which can be found within your GTPE profile .

Withdrawals, Substitutions, and Transfers

Course registration changes.

Please see our  Terms and Conditions  for complete details on our policies for course registration changes.

Transcripts, Certificates, and Credits

Are your ceus accredited.

GTPE’s use of CEU follows accepted criteria and guidelines established by the Georgia Board of Regents which follows international standards such as The International Association for Continuing Education and Training (IACET).

Do you provide transcripts or certificates for Professional Development Hours or Professional Development Units?

GTPE does not issue transcripts or certificates with Professional Develop Hours (PDH) or Professional Development Units (PDU), but the crosswalk here is provided for reference.

One CEU = 10 contact hours of instruction One PDH = 1 contact hour of instruction (one CEU = 10 PDH) One PDU = 1 contact hour of instruction (one CEU = 10 PDU)

Will I receive a course completion certificate?

Upon successful completion of most GTPE courses (80% minimum attendance and a passing grade in courses that require an assessment), you may receive a certificate indicating the number of CEUs earned. Certificate issuance exceptions include courses with outstanding credentialing entities (i.e. OSHA or PADI).

How do I request a transcript of my CEUs?

CEUs earned are recorded in the attendee’s name and will appear on a GTPE transcript. All transcripts must be requested by the attendee via the transcript request form . Requests are typically processed within three business days.

How do I petition for a program certificate audit?

For an audit of your transcript for progress toward completion of a certificate, please complete the transcript request form . GTPE courses do not provide academic or degree credit. Georgia Tech academic or degree credit is only available to matriculated students taking courses that meet degree requirements.

What requirements are required by my state and association?

For specific information on state licensing or credit requirements, please contact your state licensing board. If you are seeking certification through a professional association, please review the specific requirements with that association.

TRAIN AT YOUR LOCATION

We enable employers to provide specialized, on-location training on their own timetables. Our world-renowned experts can create unique content that meets your employees' specific needs. We also have the ability to deliver courses via web conferencing or on-demand online videos. For 15 or more students, it is more cost-effective for us to come to you.

Flexible Schedule

Group training, customize content, on-site training, earn a certificate, want to learn more about this course.

Understanding Design of Experiments (DoE) in the Pharmaceutical Industry

12/13 October 2021, Heidelberg, Germany

Course No. 18390

Dr. Raphael Bar

BR Consulting

This course will explain the basics of DoE with practicing with factorial and fractional DoE as well as DoE by RSM If you have no or little previous knowledge with DoE, you will learn how to set up an experimental design and how to explore the effect of factors that influence either a development/production process or an analytical procedure  while taking into account interactions between the factors.   To better understand and assimilate the DoE  principles, you will learn first to calculate the main effects and factors  interactions by simple manual calculations (with Excel). Then, you will learn how to use Minitab  software program to create a variety of DoE designs, analyze and interpret them. Multiple exercises and examples from pharmaceutical development and laboratory analysis such as robustness studies will be solved by the participants. The participants will learn how to interpret the output of a DoE programme.

With FDA´s Process Validation Guidance for Industry from 2011 and the Annex 15 Revision 2015 process validation has changed to a life cycle. And the life cycle starts with the development which delivers process knowledge and the critical process parameters. To get there the FDA mentions „Design of Experiments“ (DoE). Therefore, DoE is a tool for implementing the process validation life cycle.   Also, ICH Guidelines Q8 (Pharmaceutical Development) and ICH Q9 (Quality Risk Management) speak about DoE as a tool, also in relation to Quality by Design (QbD= approaches).   Meanwhile, DoE is also common practice in other pharmaceutical areas, i.e. in the analytical development or as a CAPA measure for process optimisation.

Target Group

The addressees of the event are employees from the development, quality control lab and  quality assurance departments who are using DoE or wanted to use DoE in the future. We address also GMP auditors and inspectors and validation personnel also involved in DoE.

Each participant should bring a laptop with Excel and a previously downloaded 30 day free-trial Minitab 19 program from http://www.minitab.com . This program should be downloaded on a laptop a few days before the beginning date of the course and verified that it works on the laptop.

Seminar Programme as PDF

• DoE and Quality by Design
• Regulations (EU and FDA)
• A factorial experiment
• DoE vs one-at-a-time experiment
• Where is DoE applied in development and validation of analytical methods
• Where is DoE applied in manufacturing process
• development and validation
• Factorial experiments (categorical and numeric factors)
• Two and three factorial designs
• Manual calculation of main effects
• Manual calculation of interactions
• What is an orthogonal DoE
• Exercises with Excel
• Basic structure of Minitab software
• Input of data
• Running a DoE
• Plotting output results
• Practicing with Minitab
• Diagnostics for goodness of fit to model
• Deviations from normality plot
• Making replicate experiments
• Adding experiments at centre points
• Using known variability
• Two factor full DoE experiments
• Interactions between two factors
• Plotting Main effects and Interactions
• Interpretation of DoE Minitab output
• Does the linear fit the model?
• Significance with p values
• General full factorial DoE
• Exercises in interpretation of Minitab outputs
• Two and three factor experiments with Minitab
• Aliasing in DoE experiments
• Resolution of  DoE experiments
• 4-7 fractional factorial DoE
• Blackett-Burmann designs
• Definitive screening design
• Robustness of HPLC method with fractional DoE
• Optimisation of a process with fractional DoE
• 22 factorial experiments with RSM
• Contour plot
• Surface plot
• Concept of Design Space
• Exercises: optimization of drug solubility with RSM design
• Effect of process parameters on dissolution assay and variability
• Why we use DoE in the pharmaceutical development?
• Example: DoE for formulation selection / optimization
• Example: DoE for manufacturing process optimization
• DoE vs “traditional” approach – when to use which?
• Screening experiments
• Fractional experiments
• Full factorial experiments
• Optimisation experiments: Surface Response Methodology
• Design Space versus Proven Operating Range (PAR)
• Normal Operating Range (NOR)
• Robustness of experiments of a process/method

This course is part of the GMP Certification Programme "ECA Certified Validation Manager" Learn more

This training/webinar cannot be booked. Send us your inquiry by using the following contact form.

To find alternative dates for this training/webinar or similar events please see the complete list of all events .

For many training courses and webinars, there are also recordings you can order and watch any time. Just take a look at the complete list of all recordings .

Please contact us: Tel.: +49 6221 8444-0 E-Mail: [email protected]

• Our Service

“Fantastic course – I really enjoyed the interactive structure & greatly appreciate social activity.”

Anthony Cummins, Sebela Pharmaceuticals, Ireland GMP Auditor Practice, September 2023

“Very well organized, information on point without being overwhelming.”

Eleni Kallinikou, Pharmathen Live Online Trainng - Pharmaceutical Contracts - Febuary 2024

“Good overview of different types of agreements, good to see both the GMP and the legal angle”

Ann Michiels, Johnson&Johnson Live Online Trainng - Pharmaceutical Contracts, Febuary 2024

“Well prepared presentations and good presenters. I also like the way of asking questions.”

Alexandra Weidler, Hookipa Biotech GmbH, Austria Live Online Training – QP Education Course Module A, November 2023

Design and Analysis of Experiments

• All Engineering
• Management Students
• Manufacturing companies like GM, Tata Motors, Tata Steel
• Process industries such as ONGC
• General Electric
• R&D organizations

--> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> -->

Note: This exam date is subjected to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout, books and references, instructor bio.

Prof. Jhareswar Maiti

Course certificate.

DOWNLOAD APP

SWAYAM SUPPORT

Please choose the SWAYAM National Coordinator for support. * :

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 June 2024

Cross shard leader accountability protocol based on two phase atomic commit

• Zhiqiang Du 1 ,
• Wendong Zhang 1 ,
• Liangxin Liu 1 &
• Yanfang Fu 1  

Scientific Reports volume  14 , Article number:  14953 ( 2024 ) Cite this article

Metrics details

• Computer science
• Information technology

Sharding blockchain is a technology designed to improve the performance and scalability of traditional blockchain systems. However, due to its design, communication between shards depends on shard leaders for transmitting information, while shard members are unable to detect communication activities between shards. Consequently, Byzantine nodes can act as shard leaders, engaging in malicious behaviors to disrupt message transmission. To address these issues, we propose the Cross shard leader accountability protocol (CSLAP), which is based on the two-phase atomic commit protocol (2PC). CSLAP employs byzantine broadcast/byzantine agreement (BB/BA) for Byzantine fault tolerance to generate cross-shard leader re-election certificates, thereby reducing the impact of shard leaders on inter-shard communication. It also uses Round-robin mechanism to facilitate leader re-election. Moreover, we demonstrate that CSLAP maintains the security and liveness of sharding transactions while providing lower communication latency. Finally, we conducted an experimental comparison between CSLAP and other cross-shard protocols. The results indicate that CSLAP exhibits superior performance in reducing communication latency.

Introduction

A blockchain 1 is essentially a decentralized database. It is a new application mode of distributed data storage 2 , 3 , point-to-point transmission, consensus mechanisms, encryption algorithms and other computer technologies 4 , 5 . However, for blockchains to be more widely used, the transaction throughput must be improved, and the confirmation delay should be decreased. Sharding is one of the key methods for solving the blockchain scalability problem, and an increasing number of systems are implementing sharding blockchains 6 , 7 . The key idea behind this approach is to create groups of nodes (or shards) that process only a subset of all transactions and system states, relying on the classic Byzantine fault tolerance protocols (BFTs) to achieve intra-shard consensus. The main purpose of intra-shard consensus is to process transactions within a shard. Therefore, the design of a consensus algorithm within a shard plays a vital role in the efficiency of transaction processing. The consensus algorithm needs to be able to efficiently process transactions within a shard and coordinate with other shards to complete cross-shard transactions, and this process requires the consensus algorithm in the shard to have a high degree of reliability and security to ensure stable network operations. Additionally, some sharding blockchains use reference committees 8 , so intra-shard consensus is also used to confirm the committee membership list. These sharding blockchain systems achieve optimal performance and scalability because: (i) Nonconflicting transactions can be processed in parallel by multiple shards; and (ii) The system can scale by adding new shards. This separation of transaction processing across shards is not completely “clean”; a transaction may depend on data managed by multiple shards, requiring the extra step of cross-shard consensus among related shards.

Atomic commit protocols (such as the two-phase atomic commit protocol (2PC)) typically operate on all relevant shards to ensure that cross-shard transactions are accepted by all or none of the shards.

In the 2PC protocol 9 , communication between shards is crucial. The 2PC protocol consists of a preparation phase and a commit phase 9 . During the preparation phase, the shard runs an in-shard consensus algorithm to prove whether the transaction input is available. We call this the availability certificate ( AC ). To reduce the complexity of communication, shards often elect a leader to act as a representative for communicating with other shards. However, centralizing trust in one node can lead to malicious behaviour. We introduce cross-transaction censorship attacks in the sharding blockchain, where an attacker can attack the leader of a related shard in a transaction, such as the BFT initiator of each shard in a Chainspace 10 ; After the shard leader is attacked and becomes a malicious leader, that malicious leader normally participates in and executes the BFT consensus algorithm within the shard (such as the PBFT, HotStuff, or Marlin); however, when cross-shard transactions are involved, the AC needs to be generated and sent to the relevant shards, and the malicious leader will not send AC to other shards related to the transaction. If the other shards do not receive all AC , they cannot agree on every part of the transaction, and the transaction will be permanently delayed. Although the intra-BFT consensus within a shard does not affect its liveness and security, the loss of agreement activity between shards still compromises the overall activity of the sharding blockchain. An attack can be executed even if the Byzantine security assumption 11 is met, and this attack wastes system resources.

There are currently two critical cross-shard consensus methods: (i) client-led cross-shard consensus and (ii) shared-led cross-shard consensus 12 . An attack on this type of consensus suits the shard-led cross-shard consensus and does not require coordination from external entities. Based on our analysis of performance trade-offs and vulnerabilities related to cross-transaction censorship in current shard-led cross-shard protocols, we propose the CSLAP protocol. The CSLAP we propose is general and applicable to sharding blockchain systems that use shard-led cross-shard consensus, such as RapidChain 13 , which utilizes the process of a 2PC to achieve high performance and scalability. RapidChain also combines the 2PC process with other methods to resist cross-transaction censorship attacks to achieve a more efficient protocol. The CSLAP identifies malicious shard leader behaviour when it does not receive the information that would allow it to perform the conversion of malicious leaders between shards and shards in the 2PC 9 . In the case of malicious behaviour, replacing the malicious leader node is the quickest solution compared to other remedial actions.

We implemented the CSLAP prototype in the Go language and compared it with other cross-transaction censorship attack defence protocols. The results show that the delay of our protocol is lower.

This study makes the following main contributions: (i) Cross-transaction censorship attacks against a shard-led cross-shard consensus protocol were studied. Then, we introduced the classic sharding blockchain protocol and the latest cross-transaction review attack defence protocol. (ii) A new CSLAP that integrates our proposed defences to achieve resilience against cross-transaction censorship attacks without compromising the 2PC protocol was designed. (iii) The Byzantine prototype was implemented, and its performance was evaluated on real distributed node sets. Then, it was compared with previous protocols such as the CSVC 14 and the CSVC in a flexible sharing blockchain protocol (FS_CSVC) 15 . The results showed that the latency of the CSLAP protocol was the lowest.

Related work

Among cross-shard blockchains, ELASTICO 16 was the first public, decentralized, shard-based blockchain system. In ELASTICO, each shard is responsible for verifying a set of transactions and reaching a consensus based on the practical Byzantine fault tolerance (PBFT) algorithm. Afterwards, the final shard verifies all of the transactions received from the shards and includes them in a global block, which is then broadcast to all of the nodes in the system and stored. In RapidChain 13 , clients no longer need to request asset proofs from each input committee. Instead, they only need to send the transaction to any committee, and then, the transaction is routed to the output committee through the inter-committee routing protocol. In the OmniLedger solution 17 , the client initiates a request to the input committee, expecting each input committee to provide proof of acceptance for its respective assets. If the client does not receive all of the asset proofs and does not unlock the command, it assumes that the shards are honest and will not fail. Eventually, all of the messages are received, and a BFT consensus is reached. OptChain 18 includes an optimal transaction placement strategy by learning past transaction patterns, while BrokerChain 19 divides the account state graph through Metis, reducing cross-shard transactions based on the account model. Both models aim to reduce the number of transactions across shards and maintain load balancing across shards by appropriately distributing transactions to shards. Chainspace 10 employs a shard-led cross-shard consensus protocol called SBAC, wherein the client submits a transaction to the input shard. Each shard internally executes a BFT protocol, which enables it to make a temporary determination regarding the local acceptance or termination of the transaction. Subsequently, the shard broadcasts its local decision (preaccept(T) or preabort(T)) to other shards through the BFT initiator.

The above protocol assumes that the final result of each shard is honest and that the shard leader is honestly sending the generated information to the relevant nodes during the cross-shard process. In general, a malicious node in a shard can change the leader by trying to switch protocols, but members in a shard are usually unaware of the information between shards. Pyramid 20 is a blockchain based on hierarchical sharding in which some nodes can store multiple shards and process cross-shard transactions involving those shards. Since nodes may be waiting for resources to be released during the block commit phase, system deadlocks can easily occur. PolyShard 21 applies coded computation to sharding for linear scalability and security. It splits data into coded shards, enabling parallel processing and recovery of damaged shards. Ren et al. 22 proposed root graph placement to identify the shards that are most suitable for transactions based on interactions between the most recent transactions and historical transaction s and designed two techniques to mitigate the impact of the remaining cross-shard transactions on the system performance. One technique parallelizes dependent transaction validation with atomic commit protocols, and the other incorporates atomic commit protocols. X-shard 23 is a cross-shard transaction scheme for blockchain systems that uses an optimistic strategy to handle low-latency multiple input multiple output (MIMO) cross-shard transactions. Transactions are assumed to be valid and only verified when they are about to be committed to the output shard. Cross-shard transactions are processed through a gateway account managed by the board. Each cross-shard transaction is broken down into multiple intra-shard sub-transactions that transfer coins to the gateway account in the input shard. Input shards process sub-transactions in parallel without waiting for other shards. If cross-shard transaction verification fails, the transaction is rolled back to ensure atomicity. The above protocols can reduce the proportion of cross-shard transactions or improve the delay they cause through parallelization , transaction merging and other technical means. As 24 noted, most of the previous studies were based on smaller variants of 2PC for cross-shard transactions.

In the description of cross-shard consensus attacks for Byzcuit 25 , which was designed by Alberto et al., the attackers resist the replay attacks of both the shard-led and client-led cross-shard consensus protocols. For cross-transaction review attacks, after the output leader does not receive a relevant shard certificate, the secure cross-shard view-change protocol 14 (CSVC) proposed by Liu et al attempts to perform intra-shard consensus to generate a cross-shard view-change certificate to replace the view. However, the protocol’s design cannot accurately detect malicious behaviour exhibited by the leader within its own shard or within an input shard. The process of re-inquiry by shard members can only ensure the normal execution of the protocol, which increases the burden in terms of delay, and malicious leadership styles cannot be accurately identified. Liu referenced and improved the cross-shard view-change protocol in a flexible sharding blockchain protocol 15 in 2023 (FS_CSVC). The two cross-preparation and cross-commit construction stages are used to generate cross-shard view-change certificates. This protocol incorporates the members processes by repeatedly asking for certificate information. However, the process of re-inquiry still exists, and the problems mentioned above still exist.

Therefore, this paper proposes the CSLAP, which reduces the query process and ensures that leaders’ malicious behaviour is detected at every step, thus reducing latency and improving security across shared consensus.

System model and overview

System model, transaction ( tx ).

tx refers to an operation or event recorded and verified on the blockchain network. Transactions typically involve the transfer of data, assets, or value, which serve as the fundamental building blocks of a blockchain system. A transaction consists of multiple inputs and outputs:

Inputs: These define the source of tx , which could be outputs from previous tx or other types of digital assets.

Outputs: These represent the destination of tx .

Honest shard

We assume that all of the shards in the protocol are honest. An honest shard means that the proportion of honest members within it meets the target security threshold determined by the shard’s internal consensus algorithm. Additionally, the shard reorganization process does not impact the assumption of shard honesty.

In a sharding environment, the input and output of a transaction can be in different shards. We refer to the shard that manages the input as the input shard ( \(shard_{in_1}\) ,..., \(shard_{in_n}\) ) and the shard that manages the output as the output shard ( \(shard_{out_1}\) ,..., \(shard_{out_n}\) ). Sometimes, if a transaction’s partial input and output are in the same shard, that shard can simultaneously be both an input and an output shard.

We indicate that \(T\{(I_1,..., I_x) \rightarrow (O_1,...,O_y),I_x\in shard_ {in_x},O_y\in shard_ {out_y} \}\) . For example, \(T\{(I_1, I_2)\rightarrow (O_1, O_2, O_3)\}\) indicates a transaction with two inputs \(I_1,I_2\) and three outputs \(O_1, O_2, O_3\) , including the input divided \(shard_ {in_1}, shard_ {in_2}\) and the output divided \(shard_ {out_1}, shard_ {out_2}, shard_ {out_3}\) , \(shard_ {in_1}\) and \(shard_ {out_1}\) , which are the same shard; however, it is easier to describe the different shard identities during the protocol operation.

Let \(R_{in}\) ={ \(r_{in_1}\) ,..., \(r_{in_i}\) ,..., \(r_{in_n}\}\) and \(R_{out}\) = { \(r_{out_1}\) , \(r_{out_2}\) ..., \(r_{out_i}\) ,..., \(r_{out_n}\}\) be the set of n nodes in \(shard_{in}\) and \(shard_{out}\) . Each node \(r_{out_i}\) or \(r_{in_i}\) has a key pair ( \(sk_i\) , \(pk_i\) ) and is uniquely identified by its public key \(pk_i\) in the system. We assume a public key infrastructure (PKI) that knows the public keys of all nodes to prevent identity spoofing.

In this system, nodes are classified based on their behaviour towards the protocol as follows:

Honest nodes: These nodes adhere to the protocol, following the predefined rules and processes with integrity. Honest nodes ensure the system’s reliability by conducting operations transparently and without deviation.

Malicious nodes: These nodes deviate from the protocol, engaging in behaviours that contradict or undermine the established rules. Malicious nodes may attempt to manipulate the system for personal gain or to disrupt the consensus process. They represent a risk to the system’s security and can cause inconsistency or data corruption.

Adversary model

An adversary can destroy f of n nodes per-shard, where \(n\ge 2f+1\) and the remaining nodes are correct. Therefore, the number of honest nodes is greater than n /2 to ensure an honest majority within each shard. The adversary is static, i.e. it chooses a set of f nodes to destroy at the beginning of the protocol and cannot choose to destroy other nodes afterwards.

Quorum certificate

There is a group of voting signatures that form the node’s quorum certificate (QC), and the QC is composed of the voting rights of nodes 1/2 or more in the shard. A QC serves as a crucial component in blockchain governance and consensus mechanisms, ensuring that an action or block has received sufficient approval from the participating nodes. It acts as a proof of majority support, validating the authenticity and consensus behind a decision or block addition.

Network model

We adopt a synchronous network model in which any message sent by an honest node at time t arrives at another node before time \(t+\Delta\) , where \(\Delta\) is the known network maximum delay. The opponent can pass the message delay to a known upper bound \(\Delta\) .

Security properties

Definition 1.

(Consistency) In sharding blockchains, honest members reach a consensus on a transaction and will not commit to two conflicting transactions.

Definition 2

(Liveness) For any valid tx submitted by a client, after a certain period, all relevant shards finally decide whether to execute accept(T) or abort(T).

Definition 3

(Cross-shard transaction censorship attack) Let \(tx=T\{(I_1,...,I_x) \rightarrow (O_1,...,O_y)\) be a cross-shard transaction where \(I^j(j\in [1,x])\in shard_{in_j}\) . Let \(L_{j}\) behave as the leader of \(shard_{in_j}\) . If \(L_{in_j}\) pretends to be honest in \(shard_{in_j}\) but does not transfer or send invalid AC between related shards, then \(L_{in_j}\) is judged to launch a cross-shard transaction censorship attack.

Cross-transaction censorship attack display.

Figure 1 briefly shows an example of an attack across attempts at censorship, depicting various situations in the 2PC phase for a tx with two inputs and three outputs.

The two inputs also serve as outputs for receiving acceptance or rejection information from the set coordinator. Here, the output \(shard_{out_3}\) acts as the coordinator. After receiving AC from the input shards \(shard_{in_1}\) and \(shard_{in_2}\) , the blockchain system performs verification and sends acceptance or rejection information back to \(shard_{in_1}\) and \(shard_{in_2}\) . Table  1 shows the cross-censorship transaction attack behaviour in the 2PC phase. Preaccept(T) means that an input shard temporarily writes changes locally; preabort(T) indicates that the write failed. Lock refers to locking a specific related object of the transaction; accept(T) means that the coordinator has received confirmation of the transaction, while abort(T) means that the coordinator has sent information indicating that the transaction has been rejected. Inactivated refers to permanently confirming changes and releasing related resource information, and unlocked refers to unlocking resources and rolling back the local writes.

Performance metrics

The CSLAP protocol involves a performance metric (the delay between finding a maliciously behaving leader and switching to an honest leader):

Latency (ms): latency for submitting cross-shard transactions.

Throughput (bytes/s): the number of bytes processed per second.

CSLAP:Cross shard leader accountability protocol

Protocol overview.

The CSLAP allows the coordinator shard \(Shard_{out}\) to observe the malicious behaviour of \(Shard_{in}\) ’s leader. Subsequently, the CSLAP generates a shard \(Shard_{in}\) through the consensus algorithm proof related to the leader’s malicious behaviour and sends it to the members of \(Shard_{in}\) . After the members of \(Shard_{in}\) judge the authenticity of the cross-shard leader re-election quorum certificate (CSLAP-QC) sent by \(Shard_{out}\) , the members of \(Shard_{in}\) use the round-robin algorithm to replace the malicious leader. Here, we use the BFT protocol based on the two primitives, BB/BA. Using a BFT protocol based on the BB requires protection against malicious behaviour by shard leaders who send cross-shard certificates. The BFT protocol based on the BA can omit the leader’s negative behaviour.

Round-robin leader election 26

The round-robin approach allows nodes to elect a leader for each epoch. The round-robin algorithm takes the last block as input and outputs a random leader from all nodes excluding the last f and misbehaving leaders.

Byzantine broadcast

A Byzantine broadcast is a protocol that allows a certain node (called the sender) to consistently distribute a message among n intra-shard nodes, in which up to f nodes are Byzantine.

Definition 4

(Byzantine Broadcast 27 ) A Byzantine broadcast satisfies the following properties:

(Agreement) If two honest nodes commit values B and \(B'\) , respectively, then B = \(B'\) .

(Termination) All honest replicas eventually commit and terminate.

(Validity) If the designated sender is honest, then all honest nodes commit on the sender’s value.

Byzantine agreement

The Byzantine agreement is a protocol that allows certain nonfaulty (honest) replicas to decide on the same intra-shard output, in which up to f nodes are Byzantine.

Definition 5

(Byzantine Agreement 27 ) The Byzantine agreement satisfies the following properties:

(Validity) If all honest replicas have the same input value, then all honest replicas commit on the value.

Definition 6

(Byzantine Fault Toleranc 28 ) The BFT is a pivotal algorithm in distributed computing systems that is designed to achieve consensus in networks in which some nodes may fail or act maliciously. The BFT addresses the challenge of ensuring that a distributed system continues to operate reliably and reaches consistent decisions even when a subset of nodes experiences faults or exhibits hostile behaviour. The BFT protocol provides the following properties.

( Safety ): Every two honest nodes do not commit different blocks at the same height.

( Liveness ): If a transaction is received by an honest node, then the transaction will eventually be included in every honest node’s ledger.

Definition 7

(Two-phase Atomic Commit Protocol 9 ) This protocol has two phases that are run by a coordinator. During the first voting phase, the nodes tentatively write changes locally, lock resources, and report their status to the coordinator. If the coordinator does not receive AC from a node, it assumes that the node’s local writes failed and sends a rollback message to all of the nodes to ensure that any local changes are reversed and that the locks are released. If the coordinator receives AC from all of the nodes, it initiates the second commit phase and sends a commit message to all of the nodes so that they can permanently write the changes and unlock resources. In the context of a sharding blockchain, the atomic commit protocol operates on shards (which make the local changes associated with the voting phase via an intra-shard consensus protocol such as the PBFT) rather than on individual nodes.

Definition 8

(Availability Certificate 9 ) In the 2PC protocol, AC is a crucial document generated by each input shard involved in a transaction. These certificates are produced at the end of the first phase of the transaction and indicate each shard’s preliminary reaction to the transaction, namely, whether to accept (preaccept) or reject (preabort) the continuation of the transaction.

Concrete protocol

Based on the above primitives, we propose our CSLAP protocol. Table  2 provides a complete protocol flow. The following describes the detailed execution process of the agreement through a transaction:

2PC-prepare

This phase is the first phase of a standard cross-shard transaction (the 2PC-prepare phase). Initially, the client broadcasts the transaction tx to all of the shard members. Then, a shard that does not contain the relevant transaction information forwards the tx information to the relevant shard and discards the transaction tx . The shard retains the transaction tx containing the relevant transaction information. Shard members then run the intra-shard consensus algorithm to generate a transaction \(AC_i\) and send it to each relevant input and output shard. The input shard \(shard_{in_i}\) is used to run the intra-consensus algorithm to generate the transaction \(AC_i\) . After \(T_{BFT}+\Delta\) , each relevant shared 1/2 member and leader should have received all \(AC_i\) related to tx .

BFT protocol based on the BB

As a coordinator shard \(shard_{out}\) , if the AC of all input shard transactions are typically received, then \(shard_{out}\) executes the second phase of 2PC. After the members of \(shard_{out}\) receive all relevant AC , they broadcast the certificate information and start a timer. The timer is set to \(t0+2\Delta\) , where t 0 is the timestamp in the last AC accepted, indicating the time when the input shards sent AC . The shards wait for the leader to take action until it times out, and they perform a view-change protocol to replace the leader. If the \(AC_i\) for an input shard is still not received, the leader \(L_{out}\) of \(shard_{out}\) votes and runs the BFT protocol based on the BB to generate a CSLAP-QC. To submit a proposal, two rounds of voting are needed. \(R_{out}\) collects \(f+1\) valid votes \(<m_{vote}>\) and uses the collected votes to build a CSLAP-QC.

BFT protocol based on the BA

As a coordinator \(shard_{out}\) , if the AC for all input shards is typically received, \(shard_{out}\) performs the 2PC-commit. If the AC for an input shard is still not received, member \(r_{out}\) in \(R_{out}\) executes the BFT protocol based on the BA phase. \(shard_{out}\) runs the BFT protocol based on the BA to generate a CSLAP-QC. After two rounds of voting, \(R_{out}\) collects \(f+1\) valid votes \(<m_{vote}>\) and uses the collected votes to construct the CSLAP-QC.

CSLAP-QC transfer

After generating the CSLAP-QC in shard \(shard_{out}\) , \(R_{out}\) sends the CSLAP-QC to all members in \(shard_{in_i}\) and starts a timer of 2 \(\Delta\) . \(R_{in_i}\) first verifies the CSLAP-QC signature; if the verification is passed, the members of \(shard_{in_i}\) agree that their leader \(L_{in_i}\) is a malicious leader, and \(L_{in_i}\) does not provide \(AC_{x}\) to other shards. The members of \(shard_{in_i}\) initiate a round-robin operation to switch the leader.

Leader re-election

When the current leader \(L_{in_i}\) is found to be acting maliciously, nodes \(R_{in_i}\) use a round-robin algorithm to elect a new leader \(L_{in_{i'}}\) . The algorithm uses the latest block in the blockchain as input to select a new leader from all nodes but excludes up to f nodes that have exhibited malicious behaviour and have recently served as leaders. The new leader \(L_{in_{i'}}\) resends the \(AC_i\) information before the block state defaults to show malicious leader behaviour.

If \(shard_{out}\) still does not receive the AC for \(shard_{in_i}\) before the 2 \(\Delta\) timer expires, then the leader replaced by \(shard_{in_i}\) is considered malicious. The CSLAP continues to execute until \(shard_{in_i}\) becomes an honest node. After receiving all of the certificate information, \(shard_{out}\) puts the transaction on the chain once the verification is completed. Additionally, it sends the acceptance or rejection information to all of the relevant shards.

Security analysis

( Consistency ) If the BA/BB and 2PC of the BFT protocol are consistent, then this agreement is considered consistent.

We assume that tx \(T\{(I_1,..., I_x)) \rightarrow (O_1,...,O_y) \}\) , where \(Shard_{out_c}\) is the coordinator shard. The transaction is broadcast by the client to all shards, and the input shard \(shard_{in_i}\) needs to generate the corresponding AC . The intra-shard BFT consensus is used to generate AC , which satisfies consistency for all required certificate stages. At this point, \(shard_{in_i}\) locks the relevant \(I_i\) and sends AC to the coordinator shard. AC contains both the preaccept and preabort signals. \(Shard_{out_c}\) includes the following cases:

If \(Shard_{out_c}\) receives preabort (T), the transaction is considered invalid. The coordinator shards execute BFT consensus and send abort(T) to all related shards, indicating that the transaction has failed. The input shard unlocks \(I_i\) . During this process, message broadcasts are sent to all shard members, ensuring that the members are informed. If a leader fails to send messages honestly, Shard members can still use the round-robin protocol to replace the leader, which also satisfies consistency. Therefore, this process satisfies consistency.

During the 2PC process, two conflicting transactions cannot be submitted simultaneously, ensuring that this protocol is immune to double-spending attacks. When \(Shard_{out_c}\) has received enough evidence indicating preaccept(T) for each shard, the transaction is considered valid. \(Shard_{out_c}\) reaches a consensus on the transaction through the BFT and broadcasts the message to all shards. The CSLAP ensures that the message is broadcast to all members of all shards, ensuring that all shards receive the message. At this point, all of the input and output shards reach a consensus regarding the final outcome of the transaction, satisfying consistency.

If the leader of \(shard_{in_i}\) is compromised and fails to send AC to some members, \(Shard_{out_c}\) runs the BFT to generate the CSLAP-QC, which is transmitted to its sender within the specified time frame. After receiving the CSLAP-QC, the input shard members save the current state for the round-robin process, ensuring that the current state in the blockchain is consistent with the pre-round-robin state. The new leader rebroadcasts AC . At this point, \(shard_{in_i}\) rebroadcasts, while \(Shard_{out_c}\) has not received all the AC , leading to a consistent waiting state for the transaction tx . The round-robin protocol does not affect the consistency property. Hence, the sharding protocol does not conflict with public prefixes and shards.

When \(Shard_{out_c}\) sends the CSLAP-QC, a 2 \(\Delta\) timer is initiated. If the timer expires without receiving AC from \(shard_{in_i}\) , the CSLAP protocol is re-executed until the AC from \(shard_{in_i}\) is received. During this process, all shards are either consistent or in a waiting state, meeting the consistency requirement.

Thus, the sharding system can maintain transaction consistency, proving the correctness of the protocol. \(\square\)

( Liveness ) For any valid transaction tx submitted by a client, after a certain period, all relevant shards finally decide to execute accept(T) or abort(T).

Let us assume that the start moment occurs when a client’s transaction request is sent to all input shards. The liveness parameters are related to the parameters of the BFT protocol. For the input to a transaction, the input shard takes \(T_{BFT}\) time to generate AC and \(\Delta\) time to send AC to the relevant shard. During the 2PC-commit phase, \(shard_{out}\) also needs \(T_{BFT}\) time to commit the transaction. It takes \(T_{BFT}+\Delta\) to provide feedback to the input shard and reach full agreement. Therefore, when the relevant leaders are honest and optimistic, the transaction confirmation parameter \(T_{liveness}\) can be calculated as \(3T_{BFT}+3\Delta\) .

There are two cases of proof of liveness.

Part I : The leaders of all relevant shards in a transaction are honest. First, assuming that the leaders of all relevant shards for a transaction tx are honest, the input shard generates the AC of \(tx^i\) through the BFT algorithm. Then, the input shard leader sends AC to the relevant shard. In \(shard_{out}\) , after receiving all AC related to tx in \(shard_{out}\) , \(shard_{out}\) runs the intra-shard consensus algorithm and submits tx to the block.

Part II : The leaders of some shards are dishonest. The second case occurs when some input shard leaders are malicious. Malicious leaders can censor transactions in three ways:

After receiving the tx value from the client during the 2PC-prepare phase, the leader of the input shard is withheld.

The input shard generates AC through the BFT protocol, but the leader of the input shard does not send AC to \(shard_{out}\) .

The input shard generates AC through the BFT protocol, but the leader of the input shard sends a conflicting AC .

Case i: The member of the input shard does not receive the 2PC preparation proposal within the \(\Delta\) time of receiving the transaction information tx and initiates the active view-change mechanism.

Case ii/iii: After \(shard_{out}\) receives tx and waits for \(T_{BFT}+\Delta\) time, the BB/BA protocol generates the CSLAP-QC and sends it to the input shard. The input shard executes the round-robin protocol to replace the leader.

In Case ii/iii, considering the best case, the input shard is replaced with an honest node after executing the round-robin protocol once, and AC is sent normally; then, the protocol execution time is \(T_{BFT}+2\Delta\) . The transaction confirmation parameter is \(T_{liveness}=4T_{BFT}+4\Delta\) . In the worst case, if the input shard executes k ( \(k<=f-1\) ) round-robin iterations before replacing it with an honest node, the shard needs to wait for \(2\Delta\) every time CSLAP is re-executed. Therefore, the protocol execution time is \(k(T_{BFT}+2\Delta )+2\Delta +3T_{BFT}\) , and the transaction confirmation parameter is \(T_{w-liveness}=(k+3)T_{BFT}+2(k+1)\Delta\) . If an adversary is able to disrupt the liveness of CSLAP , it must either disrupt the voting power in the network by more than 1/2 or disrupt the termination of the BFT protocol based on the BB/BA, which has only negligible probability. Thus, the sharding system can maintain transaction liveness, proving the protocols correctness.

\(\square\)

In the synchronous network model, assuming that multiple malicious behaviours exhibited by different shard leaders are detected by different shards in the shard blockchain, the leaders of all malicious shards are still replaced in \(T_{liveness}\) .

This protocol allows parallel processing of malicious leaders behaviour, assuming that in a transaction, the leader of an honest shard detects the malicious behaviour of one or more leaders of the related shards. Since generating a CSLAP-QC certificate does not involve proposing blocks or generating any sequence of actions, the leader can simultaneously propose the BFT for malicious behaviour on multiple shards. After the certificate is generated, the shard members send the certificate to the relevant shards. Therefore, regardless of whether several leaders exhibit malicious behaviour, they will be the current malicious leaders within \(T_{liveness}\) . \(\square\)

Starting with sending AC t 0, the coordinator shard members detect the malicious behaviour of this shard within the maximum t 0+ \(2\Delta\) time and initiate a view change to switch and replace the current leader.

Assuming a transaction, after generating AC , the time when the shard leader sends AC is t 0, and the leader sends AC to the \(f+1\) members of the coordinator shard at the same time. The time when the members start receiving AC is t . AC contains the t 0 timestamp. The time at which the shared leader receives AC is \(t'\) , \(t0\) ( \(t,t'\) ) \(t0\) + \(\Delta\) . Assuming that the shard leader is honest, the shard members receive AC within \(t'\) + \(\Delta\) and t 0+ \(\Delta <t'\) + \(\Delta <t0\) + \(2\Delta\) . Assuming that the coordinator shard leader is malicious, the coordinator shard members will not receive the prepare information from the coordinator shard leader within \(t'\) + \(\Delta\) . If \(t\) \(t'\) , then \(t'\) + \(\Delta <t\) + \(\Delta\) \(<t0\) + \(2\Delta\) ; if \(t<t'\) , then t + \(\Delta <t'\) + \(\Delta <t0\) + \(2\Delta\) . \(\square\)

Efficiency analysis

Time complexity.

From the moment malicious behaviour is detected, if one round-robin is executed to replace the input shard with an honest node, then the protocol execution time is \(T_{BFT} + \delta\) , where \(\delta\) represents the actual transmission delay in the network. In the worst-case scenario, if the input shard needs a total of k rounds of polling (where \(k <= f - 1\) ) before switching to an honest leader, and if each time the CSLAP is re-executed, it must wait for the maximum message round-trip delay of \(2\Delta\) , then the time for k rounds of polling is \(k \times (T_{BFT} + 2\Delta )\) .

The final successful replacement takes \(T_{BFT} + 2\delta\) . Hence, the total execution time for the protocol is \((k + 1) \times T_{BFT} + 2k \times \Delta + \delta\) .

Regarding the time complexity of the BFT, considering the PBFT as an example, its asymptotic time complexity is \(O(n^2 \times \Delta )\) , where n is the number of members in a single shard. Therefore, the execution time complexity can be represented as \(O((k + 1) \times (n^2 \times \Delta ) + 2k \times \Delta + \delta )\) . Given that \(k <= f - 1\) and \(f <= \frac{n - 1}{2}\) , we have the following:

Since \(2\delta\) is constant and \(2k \times \Delta\) is smaller than \((\frac{n - 1}{2}) \times (n^2 \times \Delta )\) , the time complexity can be understood as \(O((\frac{n - 1}{2}) \times (n^2 \times \Delta ))\) .

In the best-case scenario, in which only one swap needs to be replaced with an honest leader, the time complexity is \(O(n^2 \times \Delta )\) . In the worst-case scenario, in which \(f - 1\) swaps are needed, the time complexity is \(O((\frac{n - 1}{2}) \times (n^2 \times \Delta ))\) . Thus, the overall asymptotic time complexity is \(O(n^3 \times \Delta )\) .

Communication complexity

When the system executes the CSLAP, the honest leader needs to broadcast, and the broadcast is one-to-many; therefore, the communication complexity is n , which is the number of members in a shard. When no message is received, the communication complexity of executing a BFT round (using the PBFT as an example) is \(O(n^2)\) . Then, the process of regenerating the certificate and sending it is a multicast process; thus, there are n members in the fragment broadcast, and the communication complexity is \(O(n^2)\) . Finally, if the other shards are polled, the communication complexity is 1. If k rounds are performed, then the total communication complexity is \(k(O(n^2+n^2+1)=O(k*n^2)\) . As described in Sect. " Time complexity ", the best communication complexity is \(O(n^2)\) , and the worst communication complexity is \(O(n^3)\) . Therefore, the asymptotic time complexity is \(O(n^3)\) .

Performance analysis

Table  3 shows a comparison of the protocols. Identifying malicious leadership represents whether a leader with malicious behaviour can be accurately identified for each cross-transaction censorship attack.

Cross-shard view-change protocol.

Cross shard leader accountability protocol.

Flexible sharding cross-shard view-change protocol.

Figure 2 shows the implementation process of the CSVC protocol, Fig. 3 is the execution process of the CSLAP, and Fig. 4 is the execution process of FS_CSVC. (We only focus on the security and consistency of transactions between shards and do not consider the transaction consensus process within shards.) In the CSVC, after leader \(L_{out_1}\) of \(S_{out_1}\) discovers the malicious behaviour of \(S_{in_1}\) , it notifies the members first, and then, the members ask \(L_{in_1}\) of \(S_{in_1}\) . If \(L_{in_1}\) provides a message to the inquiring member of \(S_{out_1}\) , then the member broadcasts AC in \(S_{out_1}\) , and the shard leader continues to process tx in the normal 2PC protocol; if \(R_{out_1}\) does not receive AC from \(L_{in_1}\) within \(2\Delta\) , it provides \(L_{out_1}\) with an agreement message, which is included in the BFT-commit phase. \(L_{out_1}\) then runs the BFT algorithm to generate a cross-shard view change (CSVC-CC). Next, \(L_{out_1}\) sends the CSVC-CC to \(f+1\) members of \(S_{in1}\) , and \(S_{in_1}\) members attempt to switch after verifying the certificate. We regard the two consensus rounds as \(T_{BFT}\) and use \(\delta\) to denote the real-time delay when a node sends information to other nodes. Therefore, leader replacement requires \(T_{BFT} +2\delta +2\Delta\) .

\(L_{out_1}\) sends vcs-prepare to the shard first, and \(R_{out_1}\) receives the vcs-prepare request \(L_{in_1}\) query. If no message is received, the shard signs vcs-prepare. \(L_{out_1}\) then constructs csv-commit after receiving \(2f+1\) signatures and generates \(csvc-CC\) after collecting \(2f+1\) , and the message is broadcast to all members of \(S_{in_1}\) to change the view. The time required for a leadership change is \(T_{BFT}+\delta +2\Delta\) .

FS_CSVC adopts two rounds of BFT consensus. \(L_{out_1}\) first sends CSVC-PREPARE to the inside of the shard, and \(R_{out_1}\) receives CSVC-PREPARE to ask \(L_{in_1}\) queries and signs CSVC-PREPARE if no message is received. \(L_{out_1}\) then constructs CSVC-COMMIT after receiving the \(2f+1\) signature and generates the \(csvc-CC\) information after collecting \(2f+1\) . Then, the information is sent to all members of \(S_{in_1}\) for a view change. The time required to replace the leader is \(T_{BFT}+\delta +2\Delta\) .

The CSLAP protocol uses \(S_{in_1}\) to send certificate information to at least \(f+1\) \(S_{out_1}\) to ensure that \(L_{in_1}\) ’s malicious behaviour can be determined without request, and \(S_{out_1}\) is sent to all members of \(S_{in_1}\) after the CSLAP-QC is generated.

At this time, all members have reached an agreement on the malicious behaviour of the leader so that the leader can be replaced directly through the round-robin protocol, and there is no need to change the view and internal state through a view change. Since the round-robin protocol directly replaces its own information, the time consumed is negligible. In this context, regardless of whether the BFT protocol is based on the BB or BA, the runtime is uniformly considered \(T_{BFT}\) . The time required for the protocol to replace a malicious leader is \(T_{BFT}+\delta\) . Thus, \(T_{CSVC}>T_{FS\_CSVC}>T_{CSLAP}\) .

Implementation & evaluation

We seek to answer the following questions through this evaluation:

What is the system’s performance for two different intra-shard BFT protocols and three cross-shard lead conversion protocols under conditions of varying transaction sizes, maximum delays, and shard counts? (" Performance evaluation of the intra-shard and cross-shard lead conversion protocols ")

What is the system’s performance when running the conventional two-phase commit protocol under conditions with different numbers of transaction rounds? (" Performance evaluation of the two-phase commit protocol ")

What is the system’s performance when, for example, executing multiple rounds of cross-shard transactions in the presence of a malicious leader in one of the input shards? (" Performance evaluation in complex networks ")

Implementation details and methodology

Experimental setup.

We deployed our implementation on a cloud instance from the Alibaba Cloud 29 . We used an ecs.g7.8xlarge instance (32 vCPUs, 64 GB of memory, and 25 GB/s network bandwidth) in our Hangzhou data centre 30 . The instance operates on a dedicated network with a network latency of no more than 20 ms, runs Windows 11 as the operating system, and uses 100 GB of SSD storage. In our experiment, the main metrics were latency (quantified in seconds) and throughput (quantified in bytes processed per second).

Implementation

We implemented a two-stage atomic commit model under the standard synchronous model. Shards mainly use BFTs based on the BB prototype to synchronize HotStuff (SYNC) 31 and reputation-based state machine replication (RBSMR) 26 for certificate information generation and final transaction confirmation. We modified the core logic of SYNC and the RBSMR by adding a cross-shard transaction model to ensure that messages sent from other shards are recognized and processed within the chip. Shards use a 2PC to interact with each other. The CSVC, FS_CSVC and CSLAP are three cross-shard lead conversion protocols. For different cases, we use the following abbreviations: When the intra-shard consensus is SYNC, there are no cross-shard lead conversion protocols between shards. In short, for SYNC, when the CSVC is used in the 2PC, it is called SYNC-CSVC; when using FS_CSVC, it is called SYNC-FS; and when the CSLAP is used, it is called SYNC-CSLAP. The same is true when the intra-shard consensus is the RBSMR. We used fmt to output content and log to record log information, and we utilized net/http to simulate each of the different nodes. Additionally, we used the sync package to achieve synchronization among multiple goroutines. Moreover, we have reused the intra-shard consensus operations from SYNC and the RBSMR. In our implementation, the transactions proposed by customers are all cross-shard transactions and are specific to \(T\{(I_1, I_2)\rightarrow (O_1, O_2, O_3)\}\) . \(Shard_{out_3}\) is the coordinator shard by default. Other shards submit AC to \(shard_{out_3}\) . The model only processes the next transaction after executing one transaction. All throughput and latency results are measured from clients of separate processes that run separately on the same virtual machine as the shard members. We ensured that the performance of sharding is not limited by the lack of transactions proposed by the client. The network topology of the system is shown in Fig. 5 .

Network topology structure.

Performance evaluation of the intra-shard and cross-shard lead conversion protocols

We first evaluated the baseline latency of a single-round 2PC protocol with fault tolerance ( \(f = 1\) ) in a synchronous setting with \(\Delta = 100\) ms. The number of members in each shard m was set to 4. Initially, we fixed the number of shard members and varied the transaction sizes, considering nine different transaction sizes ranging from 400 to 2000 bytes. For each transaction, we compared the latency of two distinct intra-shard consensus protocols, a standard two-phase commit protocol and cross-shard lead conversion protocols in the presence of a malicious leader. Each data point represents the latency from when the client broadcasts the inter-shard transaction to when the transaction is confirmed. After the client broadcasts the transaction, different shards reach a consensus, lock the transaction, and provide AC to the coordinating shard. As shown in Fig.  6 , under honest shard conditions, the latencies for SYNC and the RBSMR were 668 ms and 956 ms, respectively; with a malicious leader, the latencies for SYNC-CSVC and SYNC-FS were approximately 1600 ms, and for RBSMR-CSVC and RBSMR-FS, they were nearly 2000 ms, with this protocol exhibiting latencies of approximately 1000 ms and 1500 ms, respectively. The presence of a malicious leader significantly increases the execution time. As the transaction size increases, the latency tends to increase; this phenomenon mainly due to the increase in packet size and the extended data transmission time.

Latency comparison of different transaction sizes.

Latency comparison of different numbers of shard members.

Latency comparison of different MAX( \(\Delta\) ) values.

Next, we fixed the transaction size and maximum transmission delay and varied the number of shards from 4 to 8. As depicted in Fig.  7 , with changes in the number of shard members, since both internal consensus protocols are synchronous network models, the time taken typically fluctuates with the number of nodes, where system fluctuations are essentially network fluctuations.

Furthermore, we fixed other variables and set the maximum network delay to 20, 35, 50, 75, 100, 200, and 500 ms. As illustrated in Fig.  8 , at \(\Delta = 20\) ms, the latency for SYNC was 188 ms, and the latencies for SYNC-CSVC, SYNC-FS, and SYNC-CSLAP were 400, 380, and 268 ms, respectively. At \(\Delta = 500\) ms, the latencies for these protocols were 3174 ms for SYNC and 7761, 7600, and 5174 ms for SYNC-CSVC, SYNC-FS, and SYNC-CSLAP, respectively. Their multiples increased from 2.12, 2.02, and 1.42 to 2.44, 2.39, and 1.63, respectively. These results demonstrate that the smaller \(\Delta\) is, the shorter the execution time of the protocols, the lower the impact of \(\Delta\) , and the greater the impact of \(\delta\) . As \(\Delta\) increases, both SYNC and the BFT-SMR have a fixed timer cost, leading to a significant increase in network latency.

Time delay of different shard members.

Throughputs of different shard members.

Finally, we present the latency and throughput variations of the system during a round of transactions when the number of shards is 3, 5, and 7. The fixed transaction size in the system is 400 bytes, with \(\Delta\) set to 100 milliseconds, and each shard has 4 members. As shown in Fig. 9 , when the number of shards is 3, the system’s latency is consistent with the above figure. Moreover, as the number of shards increases, the latency fluctuation remains within the 10 ms range due to the synchronous network model, in which the system exhibits fixed latency behaviour during execution. Fig. 10 shows that when the number of shards is 3, the throughput of SYNC is 580 bytes/s, while those of SYNC-CSVC and SYNC-FS are 243 and 241 bytes/s, respectively. SYNC-CSLAP achieves 375 bytes/s. The throughput of the RBSMR is 426.744 bytes/second, while those of the RBSMR-CSVC, RBSMR-FS, and RBSMR-CSLAP are 195, 200, and 267 bytes/s, respectively. We observe that the throughput of the RBSMR-CSLAP is 50% and 35% higher than that of the other two protocols. As the number of shards increases, there are no changes in latency or throughput, which is attributed to the synchronous network model since it causes the system to exhibit consistent latency behaviour during execution.

In summary, in our system environment, the regular latency for a 2PC based on two intra-shard consensus protocols ranges between 660 and 960 ms, and protocols requiring leader changes result in greater latencies. The execution time for both the CSVC and FS_CSVC is three times that of the standard 2PC protocol. However, for regular submissions, the protocol requires only 1.5 times the latency. Compared to the CSVC and FS-CSVC, this protocol offers a performance improvement of approximately 50%.

Performance evaluation of the two-phase commit protocol

Next, we evaluated the latency consumed by a normal 2PC. We believe that after executing a cross-shard transaction, we can continue to execute the next transaction. All nodes in the system are honest nodes and implement the agreement as soon as they receive the message. Therefore, this test assesses the time delay and throughput of the system in the honest case. Theoretically, a normal round of two-stage atomic commits needs to go through \(3T_{BFT}+3\delta\) . Since \(\Delta\) is set to 100 ms, the transaction size is 400. As shown in Sect. " Performance evaluation of the intra-shard and cross-shard lead conversion protocols ", the SYNC latency is 660 ms, while the RBSMR latency is 970 ms (Fig. 9 ). Therefore, the sync throughput is 600 bytes/s, and the RBSMR is 410 bytes/s. We began to record the delay and throughput of the number of rounds executed (1, 5, 10, 20, 50, 100, and 200).Fig.  11 shows that the delay consumed by the system execution process is relatively stable overall, and it has little effect on factors such as network delay. It can be seen from Fig. 12 that the initial throughput was close to the throughput limit, but with the operation of the system, the network fluctuated. As a result, the data remained at approximately 575 bytes/s and 400 bytes/s.

Delay of the two-phase atom submission transaction argument.

Throughput of the two-phase atomic submission transaction argument.

Performance evaluation in complex networks

Sections " Performance evaluation of the intra-shard and cross-shard lead conversion protocols " and " Performance evaluation of the two-phase commit protocol " illustrate the execution processes of the 2PC protocol under basic conditions. Next, we aim to measure the latency and throughput of the system when executing cross-shard lead conversion protocols in the presence of malicious nodes. We set \(Shard_{in_1}\) to contain 20 members, 9 of which were malicious. The other shards remained honest. These malicious nodes, once elected as leaders, may engage in withholding behaviours. We then evaluated the system’s throughput and latency over 5, 10, and 20 rounds, as depicted in Figs. 13 and 14 .

Time delay of different trading rounds.

Throughputs of different trading rounds.

From a latency perspective, the system maintains the same delay for one round, as shown in Sect. " Performance evaluation of the intra-shard and cross-shard lead conversion protocols ", but the overall ratio of latency across 5 rounds slowly approaches similarity. For instance, SYNC-CSVC exhibited a latency of 1600 ms in one round, while SYNC-CSLAP exhibited a latency of 100 ms. After 20 rounds, the total elapsed time was 15234 ms and 14165 ms, respectively, with the performance improvement decreasing from 1.5 to approximately 1.07 times. Similar patterns are observed with other protocols.

In terms of the throughput, in a single round, SYNC-CSVC, SYNC-FS, SYNC-CSLAP, the RBSMR-CSVC, the RBSMR-FS, and the RBSMR-CSLAP have throughputs of 249, 251, 374, 199, 200, and 272 byte/s, respectively. After 20 rounds, these values increase to 525, 525, 564, 373, 373, and 392 bytes/s, respectively.

These results indicate that malicious behaviour affects the overall system latency and throughput. However, as the number of rounds increases, the throughput gradually increases and approaches the performance levels of the previous honest experiments. This improvement occurs because malicious leaders are identified and replaced promptly, with the only performance loss occurring during the replacement process. In protocols such as SYNC and the RBSMR, nodes identified with malicious behaviour are recorded and prevented from being re-elected as leaders. Our protocol demonstrates faster throughput recovery in the presence of malicious nodes; in scenarios with one round of malicious activity, the throughput is approximately 50% higher than that of the other two protocols, demonstrating less impact from malicious leadership.

We have defined a cross-shard transaction censorship attack, which threatens the liveness of shard-led cross-shard consensus protocols. This type of attack involves a malicious leader acting honestly within a shard but concealing information between shards. Additionally, the complexity of communication protocols between shards plays a significant role in the efficiency of consensus protocols. To address these issues, we propose the CSLAP, a low-latency consensus protocol designed to resist cross-shard transaction censorship.

We implemented a prototype of the CSLAP and compared it with previous protocols, and the results show that the CSLAP not only resists cross-shard transaction censorship attacks but also has lower communication latency than other protocols. This design is intended to address malicious behaviour in cross-shard consensus, primarily ensuring trusted interactions between shards. Thus, the protocol can be applied to protocols using atomic commits for cross-shard communication.

This paper mainly focuses on identifying and replacing malicious leaders in cross-shard transactions, and it uses the synchronous network model. The message arrives at the maximum network propagation delay, and the network conditions in real life may not be ideal. Therefore, in the nonsynchronous network model, the honest node may be switched due to the timing of resending cross-shard leader-re-election certificates. Second, the BFT-SMR based on the BA cannot meet the safety and activity requirements when the majority of nodes are not honest 32 . The BB-based BFT-SMR cannot satisfy safety and activity requirements with constant rounds 33 . Although we have significantly reduced the time needed to replace malicious leaders, this process may require multiple broadcasts and multicasts, increasing the complexity of inter-shard communication. Additionally, the leader replacement process might continue until an honest node is found. In rare cases, there might be consecutive malicious nodes, leading to a significant increase in delay.

In terms of potential impact and applications, our work provides new insights and solutions for the design and implementation of sharding blockchain systems. Our methods and findings can guide the design and development of future sharding blockchain systems, enhancing their security.

Future work

The experiments conducted in this paper focus on simulating the process of a single protocol. In future work, we aim to deploy this process in a complete sharding blockchain system or apply other mechanisms to ensure correct transaction confirmation even in the presence of multiple consecutive malicious leaders. We also plan to adapt the CSLAP protocol to a partially synchronous network model, enhancing its stability and reliability under varying network conditions.

Because this study primarily explores communication among malicious leaders and since the protocol only adds some additional measures during inter-shard communication without altering the fundamental process of cross-shard communication, the CSLAP may be compatible with most atomic commit protocols. In the future, we can integrate this protocol with other cross-shard communication technologies.

A potential challenge in adapting the CSLAP to different network models is managing latency variations. Strategies for addressing this issue might involve introducing adaptive timeouts that adjust based on observed network conditions. This approach would help maintain system stability even in unpredictable environments.

With respect to adversary models, we will explore mechanisms for detecting and mitigating attacks from highly adaptive adversaries, and techniques such as randomized protocol steps or frequent leader rotation could improve the system’s resilience against targeted attacks. This research direction aims to increase the robustness and adaptability of the CSLAP in diverse blockchain applications.

Overall, these future research efforts will contribute to enhancing the versatility of the CSLAP and provide innovative solutions for developing resilient blockchain technologies.

Data availability

The code sets used and/or analysed in this study are available from the corresponding authors.

Code availability

The code for this protocol is open source and available on github: https://github.com/zwd444456/CSLAPprotocol .

Whig, P. Blockchain revolution: Innovations, challenges, and future directions. Int. J. Mach. Learn. Sustain. Dev. 5 , 16–25 (2023).

Google Scholar  

Chang, F. et al. Bigtable: A distributed storage system for structured data. ACM Trans. Comput. Syst. https://doi.org/10.1145/1365815.1365816 (2008).

Article   Google Scholar  

Souri, A., Rahmani, A. M., Navimipour, N. J. & Rezaei, R. A symbolic model checking approach in formal verification of distributed systems. Hum. Centric Comput. Inf. Sci. 9 , 4 (2019).

Kim, T., Jung, I. Y. & Hu, Y. Automatic, location-privacy preserving dashcam video sharing using blockchain and deep learning. Hum. Centric Comput. Inf. Sci. 10 , 36 (2020).

Lee, Y., Rathore, S., Park, J. H. & Park, J. H. A blockchain-based smart home gateway architecture for preventing data forgery. Hum. Centric Comput. Inf. Sci. 10 , 9 (2020).

Liu, Y. et al. Building blocks of sharding blockchain systems: Concepts, approaches, and open problems. Comput. Sci. Rev. 46 , 100513. https://doi.org/10.1016/j.cosrev.2022.100513 (2022).

Article   MathSciNet   Google Scholar  

Yu, G. et al. Survey: Sharding in blockchains. IEEE Access 8 , 14155–14181. https://doi.org/10.1109/ACCESS.2020.2965147 (2020).

Arslanian, H. Ethereum (Springer International Publishing, 2022).

Book   Google Scholar  

Gray, J. N. Notes on Data Base Operating Systems (Springer, 1978).

Al-Bassam, M., Sonnino, A., Bano, S., Hrycyszyn, D. & Danezis, G. Chainspace: A sharded smart contracts platform. Preprint at http://arxiv.org/abs/1708.03778 (2017).

Altarawneh, A., Skjellum, A. The security ingredients for correct and byzantine fault-tolerant blockchain consensus algorithms. In International symposium on networks, Computer and Commununications (ISNCC) 1–9 , 2020. https://doi.org/10.1109/ISNCC49221.2020.9297326 (2020).

Bano, S. et al . Sok: Consensus in the age of blockchains. In Proc. of the 1st ACM Conference on Advances in Financial Technologies , AFT ’19, 183–198, https://doi.org/10.1145/3318041.3355458 (2019).

Zamani, M., Movahedi, M. & Raykova, M. Rapidchain: Scaling blockchain via full sharding. In Proc. of the 2018 ACM SIGSAC Conference on Computer and Communications Security , abc, 931–948, (Association for Computing Machinery, 2018) https://doi.org/10.1145/3243734.3243853 .

Liu, Y., Liu, J., Hei, Y., Xia, Y. & Wu, Q. A secure cross-shard view-change protocol for sharding blockchains. In Information Security and Privacy (eds Liu, Y. et al. ) 372–390 (Springer International Publishing, 2021).

Chapter   Google Scholar  

Liu, Y. et al. A flexible sharding blockchain protocol based on cross-shard byzantine fault tolerance. IEEE Trans. Inf. Forensics Secur. 18 , 2276–2291. https://doi.org/10.1109/TIFS.2023.3266628 (2023).

Luu, L. et al . A secure sharding protocol for open blockchains. In Proc. of the 2016 ACM SIGSAC Conference on Computer and Communications Security , CCS ’16, 17–30, https://doi.org/10.1145/2976749.2978389 (2016).

Kokoris-Kogias, E. et al . Omniledger: A secure, scale-out, decentralized ledger via sharding. In 2018 IEEE Symposium on Security and Privacy (SP) , 583–598, https://doi.org/10.1109/SP.2018.000-5 (2018).

Nguyen, L. N., Nguyen, T. D. T., Dinh, T. N. & Thai, M. T. Optchain: Optimal transactions placement for scalable blockchain sharding. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) , 525–535, https://doi.org/10.1109/ICDCS.2019.00059 (2019).

Huang, H. et al . Brokerchain: A cross-shard blockchain protocol for account/balance-based state sharding. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications , 1968–1977, https://doi.org/10.1109/INFOCOM48880.2022.9796859 (2022).

Hong, Z., Guo, S., Li, P. & Chen, W. Pyramid: A layered sharding blockchain system. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications , 1–10 (IEEE, 2021).

Li, S. et al. Polyshard: Coded sharding achieves linearly scaling efficiency and security simultaneously. IEEE Trans. Inf. Forensics Secur. 16 , 249–261 (2020).

Article   ADS   Google Scholar  

Ren, L., Ward, P. A. S. & Wong, B. Toward reducing cross-shard transaction overhead in sharded blockchains. In Proc. of the 16th ACM International Conference on Distributed and Event-Based Systems , DEBS ’22, 43–54, https://doi.org/10.1145/3524860.3539641 (Association for Computing Machinery, 2022).

Xu, J., Ming, Y., Wu, Z., Wang, C. & Jia, X. X-shard: Optimistic cross-shard transaction processing for sharding-based blockchains. IEEE Trans. Parallel Distrib. Syst. 35 , 548–559. https://doi.org/10.1109/TPDS.2024.3361180 (2024).

Ruan, P. et al . Blockchains vs. distributed databases: Dichotomy and fusion. In Proc. of the 2021 International Conference on Management of Data , 1504–1517 (2021).

Sonnino, A., Bano, S., Al-Bassam, M. & Danezis, G. Replay attacks and defenses against cross-shard consensus in sharded distributed ledgers. In 2020 IEEE European Symposium on Security and Privacy (EuroS &P) , 294–308, https://doi.org/10.1109/EuroSP48549.2020.00026 (2020).

Huang, M., Han, R., Du, Z., Fu, Y. & Liu, L. Reputation-based state machine replication. In 2022 IEEE 21st International Symposium on Network Computing and Applications (NCA) , vol. 21, 225–234, https://doi.org/10.1109/NCA57778.2022.10013518 (2022).

Abraham, I., Nayak, K., Ren, L. & Xiang, Z. Byzantine agreement, broadcast and state machine replication with near-optimal good-case latency. Preprint at http://arxiv.org/abs/2003.13155 (2020).

Lashkari, B. & Musilek, P. A comprehensive review of blockchain consensus mechanisms. IEEE Access 9 , 43620–43652 (2021).

Alibaba cloud. https://alibabacloud.com/ .

Alibaba cloud general-purpose instance families. https://www.alibabacloud.com/help/en/elastic-compute-service/latest/general-purpose-instance-families .

Abraham, I., Malkhi, D., Nayak, K., Ren, L. & Yin, M. Sync hotstuff: Simple and practical synchronous state machine replication. In 2020 IEEE Symposium on Security and Privacy (SP) , 106–118, https://doi.org/10.1109/SP40000.2020.00044 (2020).

Abraham, I., Nayak, K., Ren, L. & Xiang, Z. Good-case latency of byzantine broadcast: a complete categorization. In Proc. of the 2021 ACM Symposium on Principles of Distributed Computing , PODC’21, 331–341, (Association for Computing Machinery, New York, NY, USA, 2021) https://doi.org/10.1145/3465084.3467899 .

Hou, R. & Yu, H. Optimistic fast confirmation while tolerating malicious majority in blockchains. In 2023 IEEE Symposium on Security and Privacy (SP) , 2481–2498, https://doi.org/10.1109/SP46215.2023.10179323 (2023).

Download references

Author information

Authors and affiliations.

Xi’an Technological University, Xian, 710021, China

Zhiqiang Du, Wendong Zhang, Liangxin Liu & Yanfang Fu

You can also search for this author in PubMed   Google Scholar

Contributions

Methodology, W.Z. and Y.F.; funding acquisition, Z.D.; investigation, W.Z. and L.L.; resources, W.Z.; validation, L.L.; writing—original draft preparation,W.Z.; writing—review and editing, Z.D.

Corresponding author

Correspondence to Yanfang Fu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Du, Z., Zhang, W., Liu, L. et al. Cross shard leader accountability protocol based on two phase atomic commit. Sci Rep 14 , 14953 (2024). https://doi.org/10.1038/s41598-024-64945-1

Download citation

Received : 31 January 2024

Accepted : 14 June 2024

Published : 28 June 2024

DOI : https://doi.org/10.1038/s41598-024-64945-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on outgoing moisture content prediction models of corn drying process based on sensitive variables, 1. introduction, 2. materials and methods, 2.1. experimental design and data collection, 2.1.1. experiment materials, 2.1.2. experiment equipment, 2.1.3. experiment design, 2.1.4. data acquisition, 2.1.5. correlation test, 2.2. screening for sensitive variables, 2.3. construction of moisture content prediction model, 2.3.1. multiple linear regression model, 2.3.2. extreme learning machine model, 2.3.3. long short-term memory neural network model, 2.3.4. model evaluation, 3.1. data pre-processing results, 3.2. multiple linear regression model results, 3.2.1. mlr established by temperature and humidity variables, 3.2.2. uve-mlr established by sensitive variables, 3.3. extreme learning machine model results, 3.3.1. elm established by temperature and humidity variables, 3.3.2. uve-elm established by sensitive variables, 3.4. long short-term memory neural network model results, 3.4.1. lstm established by temperature and humidity variables, 3.4.2. uve-lstm established by sensitive variables, 4. discussion, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest, abbreviations.

Course Status :	Completed
Course Type :	Elective
Duration :	12 weeks
Category :
Credit Points :	3
Postgraduate
Start Date :	18 Jan 2021
End Date :	09 Apr 2021
Enrollment Ends :	01 Feb 2021
Exam Date :	24 Apr 2021 IST

Classification	Abbreviation	Explanation
Technical term	MLR	Multiple Linear Regression
	ELM	Extreme Learning Machine
	LSTM	Long Short-Term Memory
	RNN	Recurrent Neural Network
	UVE	Uninformative Variable Elimination
Newly defined methods	UVE-MLR	A modelling approach combining UVE and MLR
	UVE-ELM	A modelling approach combining UVE and ELM
	UVE-LSTM	A modelling approach combining UVE and LSTM

• Zhou, M.; Sun, H. Experimental study on corn grain crushing via quasi-static compression method. Trans. Chin. Soc. Agric. Eng. 2024 , 40 , 289. [ Google Scholar ]
• Wang, Z.; Wang, T.; Wu, Y.; Jingsheng, L.; Xiuying, X.; Chengbin, Z. Effects of Microwave intermittent Drying on Physicochemical Properties of Starch in Corn Kernels. J. Chin. Inst. Food Sci. Technol. 2024 , 24 , 179–190. [ Google Scholar ]
• Han, F.; Wu, W.; Liu, Z. Simulation Experiment System of Grain Continuous Drying Process and Process Control. Sci. Technol. Cereals Oils Foods 2023 , 31 , 83–89. [ Google Scholar ]
• Šooš, Ľ.; Urban, F.; Čačková, I.; olláth, Ľ.; Mlynár, P.; Čačko, V.; Bábics, J. Analysis of Thermodynamic Events Taking Place during Vacuum Drying of Corn. Sustainability 2024 , 16 , 879. [ Google Scholar ] [ CrossRef ]
• Adam, E.; William, L.; Greg, D.; Dyck, J. Performance evaluation of a non-equilibrium model for low temperature grain drying and simulation of seasonal dryer operation. Dry. Technol. 2022 , 40 , 835–851. [ Google Scholar ]
• Jibril, A.N.; Zhang, X.; Wang, S.; Bello, Z.A.; Henry, I.I.; Chen, K. Far-infrared drying influence on machine learning algorithms in improving corn drying process with graphene irradiation heating plates. J. Food Process Eng. 2024 , 47 , 14603. [ Google Scholar ] [ CrossRef ]
• Subrot, S.P.; Kaushik, L.; Singh, C.B.; Atungulu, G.; Corscadden, K. On-farm grain drying system sustainability: Current energy and carbon footprint assessment with potential reform measures. Sustain. Energy Technol. Assess. 2023 , 60 , 103430. [ Google Scholar ]
• Lin, Z.; Wang, D.; Liu, G. Application of Emulsified Fuel Oil Instead of Coal in Corn Dryer System. GRAIN Storage 2023 , 52 , 50–52. [ Google Scholar ]
• Amjad, W.; Chen, Z.; Ambrose, K. Design assessment of grain inverters in cross-flow grain dryer via CFD-DEM numerical simulation. Biosyst. Eng. 2024 , 239 , 147–157. [ Google Scholar ] [ CrossRef ]
• Bertotto, M.M.; Gastón, A.; Sánchez Sarmiento, G.; Gove, B. Effect of drying conditions on the quality of IRGA 424 rice. J. Sci. Food Agric. 2019 , 99 , 1651–1659. [ Google Scholar ] [ CrossRef ]
• Xin, L.; Kaimin, Y.; Yuancheng, W.; Du, X. Simulation study on coupled heat and moisture transfer in grain drying process based on discrete element and finite element method. Dry. Technol. 2023 , 41 , 2027–2041. [ Google Scholar ]
• Bi, Q. Multi-Field Coupled Hot Air Drying Characteristics of Multilayer Corn Seeds. Master’s Thesis, Northeast Electric Power University, Jilin, China, 2021. [ Google Scholar ]
• Hernández-Pérez, J.; García-Alvarado, M.; Trystram, G.; Heyd, B. Neural networks for the heat and mass transfer prediction during drying of cassava and mango. Innov. Food Sci. Emerg. Technol. 2003 , 5 , 57–64. [ Google Scholar ] [ CrossRef ]
• Hoon, K.; Seok, T.K.; Gwan, D.S.; Han, J.W. Optimization of Circulation Cross-Flow Dryer by Simulation During Wheat Drying. J. Biosyst. Eng. 2024 , 49 , 89–99. [ Google Scholar ]
• Çelik, E.; Koç, M.A.; Parlak, N.; Çay, Y. Prediction of the capacitance of the corn drying process parameter using adaptive- neuro-fuzzy intelligent technique with experimental validation. Dry. Technol. 2024 , 42 , 90–113. [ Google Scholar ] [ CrossRef ]
• Halaly, R.; Tsur, E.E. Continuous adaptive nonlinear model predictive control using spiking neural networks and real-time learning. Neuromorphic Comput. Eng. 2024 , 4 , 024006. [ Google Scholar ] [ CrossRef ]
• Ghosh, R.; Sahu, B.; Dey, A.; Thota, H.K.; Das, K. Artificial neural network-based approach for prediction of nanomechanical properties of anodic coating on additively manufactured Al–10Si–Mg alloy. Model. Simul. Mater. Sci. Eng. 2024 , 32 , 055006. [ Google Scholar ] [ CrossRef ]
• Xu, R.; Liang, X.; Qi, J.; Li, Z.Y.; Zhang, S.S. Advances and Trends in Extreme Learning Machine. Chin. J. Comput. 2019 , 42 , 1640–1670. [ Google Scholar ]
• Tongyu, X.; Simin, X.; Fenghua, Y.; Zhonghui, G.; Yadi, L. A BAS-ELM inversion method of japonica rice canopy nitrogen content based on a combination of multiple vegetation indices. J. Shenyang Agric. Univ. 2021 , 52 , 577–585. [ Google Scholar ]
• Jin, Y. Grain Drying Process Based on Equivalent Accumulated Temperature. Ph.D. Thesis, Jilin University, Changchun, China, 2019. [ Google Scholar ]
• Wu, Y.; Fu, D.; Yin, H. Establishment of Mathematical Model of Accumulated Temperature of Corn Based on Multi–Parameter Controllable Thin—Layer Drying Experiment and Its Tool Chart. J. Chin. Cereals Oils Assoc. 2020 , 35 , 114–120. [ Google Scholar ]
• Wang, H.; Liu, G.; Lin, L. Predictive control model of corn drying process based on neural network. Cereals Oils 2021 , 34 , 37–40. [ Google Scholar ]
• Lei, D.; Fu, Y.; Jin, H. Study on Prediction of Corn Drying Moisture Content Based on BP Neural Network. Grain Process. 2022 , 47 , 45–48. [ Google Scholar ]
• Dajbych, O.; Kabutey, A.; Mizera, Č.; Herák, D. Investigation of the Effects of Infrared and Hot Air Oven Drying Methods on Drying Behaviour and Colour Parameters of Red Delicious Apple Slices. Processes 2023 , 11 , 3027. [ Google Scholar ] [ CrossRef ]
• Xing, S.; Gao, X.; Lin, Z. A Model for Predicting the Outgoing Moisture Content of Corn Drying System Based on Extreme Learning Machine. J. Shenyang Agric. Univ. 2023 , 54 , 619–626. [ Google Scholar ]
• Kirtika, S.; Surendra, K.; Panwar, N.L.; Patel, M.R. Influences of a novel cylindrical solar dryer on farmer’s income and its impact on environment. Environ. Sci. Pollut. Res. Int. 2022 , 29 , 78887–78900. [ Google Scholar ]
• Meng, F. Numerical Simulation and Experimental Study of Moisture Heat Transfer During Corn Drying. Master’s Thesis, Henan University of Technology, Zhengzhou, China, 2022. [ Google Scholar ]
• Wang, H. Application of Regression Analysis in Modeling Drying Process of Counter and Current Grain Dryer. Mod. Food 2022 , 28 , 5–8. [ Google Scholar ]
• Wu, S.; Ren, G.; Zhang, Y. Simulation and experiment of heat and moisture transfer during ventilated drying process in maize grain pile. Trans. Chin. Soc. Agric. Eng. 2024 , 4 , 1–12. [ Google Scholar ]
• Jin, Y.; Xie, H.; Yin, J.; Zhang, Z. Research on Intelligent Control Method of Grain Drying Based on LSTM-MPC. Sci. Technol. Cereals Oils Foods 2023 , 31 , 25–34. [ Google Scholar ]
• An, N.N.; Sun, W.; Li, D.; Wang, L.J.; Wang, Y. Effect of microwave-assisted hot air drying on drying kinetics, water migration, dielectric properties, and microstructure of corn. Food Chem. 2024 , 455 , 139913. [ Google Scholar ] [ CrossRef ]
• GB 5009.3-2016 ; National Standard for Food Safety Determination of Moisture in Foods. National Health Commission of the People’s Republic of China: Beijing, China, 2016.
• Zhang, X.; Yu, B. Beijing Urban Haze Control Strategy Based on Pearson Correlation Analysis and BP Neural Network. Syst. Eng. 2023 , 41 , 26–34. [ Google Scholar ]
• Domingo, D.; Kareem, A.B.; Okwuosa, C.N.; Custodio, P.M.; Hur, J.W. Transformer Core Fault Diagnosis via Current Signal Analysis with Pearson Correlation Feature Selection. Electronics 2024 , 13 , 926. [ Google Scholar ] [ CrossRef ]
• Lotfi, M.; Chamjangali, M.A.; Mozafari, Z. Ridge regression coupled with a new uninformative variable elimination algorithm as a new descriptor screening method: Application of data reduction in QSAR study of some sulfonated derivatives as c-Met inhibitors. Chemom. Intell. Lab. Syst. 2023 , 232 , 104714. [ Google Scholar ] [ CrossRef ]
• Qu, G.; Chen, Z.; Zhang, Q. Study on germination rate of rice seed based on uninformation variable elimination method. Jiangsu, J. Agric. Sci. 2019 , 35 , 1015–1020. [ Google Scholar ]
• Centner, V.; Massart, D.L.; de Noord, O.E.; de Jong, S.; Vandeginste, B.M.; Sterna, C. Elimination of uninformative variables for multivariate calibration. Anal. Chem. 1996 , 68 , 3851–3858. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Leng, J.; Gao, X.; Zhu, J. Application of multiple linear regression statistical forecasting models. Stat. Decis. 2016 , 7 , 82–85. [ Google Scholar ]
• McIntosh, M.J. Conservative sample size for multiple regression models. Commun. Stat.-Theory Methods 2023 , 52 , 5527–5533. [ Google Scholar ] [ CrossRef ]
• Huang, G.; Zhu, Q.; Chee-Kheong, S. Extreme learning machine: Theory and applications. Neuro-Computing 2006 , 70 , 489–501. [ Google Scholar ] [ CrossRef ]
• Umair, R.U.J.; Pozarlik, A.K.; Gerrit, B. Experimental analysis of spray drying in a process intensified counter flow dryer. Dry. Technol. 2022 , 40 , 3128–3148. [ Google Scholar ]
• Zhang, M.; Yuan, M.Z.; Dai, S.S.; Chen, M.L.; Incecik, A. LSTM RNN-based excitation force prediction for the real-time control of wave energy converters. Ocean. Eng. 2024 , 306 , 118023. [ Google Scholar ] [ CrossRef ]
• Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997 , 9 , 1735–1780. [ Google Scholar ] [ CrossRef ]
• Wang, D.; Liu, G.; Zhou, G. Current Status of Rice Storage and Drying in Liaoning Province. Mod. Food 2022 , 28 , 226–228. [ Google Scholar ]
• Zhao, B. Simulation Experiment System of Grain Continuous Drying Process and Process Control. Master’s Thesis, Jilin University, Changchun, China, 2012. [ Google Scholar ]
• Lv, W.; Zhang, M.; Wang, Y.; Adhikari, B. Online measurement of moisture content, moisture distribution, and state of water in corn kernels during microwave vacuum drying using novel smart NMR/MRI detection system. Dry. Technol. 2018 , 36 , 1592–1602. [ Google Scholar ] [ CrossRef ]
• Xie, H.; Jin, Y.; Zhang, Z. Prediction and Optimization of Grain Dryer Outlet Moisture Content Based on LSTM. J. Chin. Cereals Oils Assoc. 2023 , 38 , 196–204. [ Google Scholar ]
• José, A.M.; Carlos, A.S.; Helmer, M. Nonlinear model of a rice drying process using neural networks. Vitae 2018 , 25 , 120–127. [ Google Scholar ]
• Myhan, R.; Jachimczyk, E.; Markowski, M. The Use of Graph Theory for Modeling and Analyzing the Structure of a Complex System, with the Example of an Industrial Grain Drying Line. Processes 2023 , 11 , 2812. [ Google Scholar ] [ CrossRef ]
• Hong, M.; Ai, P.; Yue, Z. Mid—Long term runoff forecasting based on FPA—ELM model in Yalong River Basin. Yangtze River 2022 , 53 , 119–125. [ Google Scholar ]
• Li, L. Research on Land Use Cover Classification Optimization Methods for Hyperspectral Remote Sensing Image Based on ELM. Master’s Thesis, Xidian University, Chengdu, China, 2018. [ Google Scholar ]
• Zhang, X.; Chang, Y. Prediction of external corrosion rate of offshore oil and gas pipelines based on FA-BAS-ELM. China Saf. Sci. J. 2022 , 32 , 99–106. [ Google Scholar ]
• Li, Y.; Chang, J.; Wang, Y. MKL-SVM algorithm for pulmonary nodule recognition based on swarm intelligence optimization. Chin. J. Eng. 2021 , 43 , 1157–1165. [ Google Scholar ]
• Li, Z. Application of population intelligence optimization algorithms to environmental monitoring problems in maize fields. J. Biotech. Res. 2024 , 16 , 77–90. [ Google Scholar ]
• Xi, F.W.; Jiang, W.Q.; Yang, M.A. Using stepwise regression to address multicollinearity is not appropriate. Int. J. Surg. 2024 , 110 , 3122–3123. [ Google Scholar ] [ CrossRef ]
• Maryam, S. Modeling survival response using a parametric approach in the presence of multicollinearity. Commun. Stat. -Simul. Comput. 2024 , 53 , 1889–1898. [ Google Scholar ]
• Abasi, S.; Minaei, S. Effect of Drying Temperature on Mechanical Properties of Dried Corn. Dry. Technol. 2014 , 32 , 774–780. [ Google Scholar ] [ CrossRef ]
• Qi, Q.; Cai, T.; Zhou, K.; Hu, Z.; Hao, F.; Wei, Y.; Ge-Zhang, S.; Cui, J. Consumers’ Risk Perception of Triploid Food: Empirical Research Based on Variance Analysis and Structural Equation Modeling. Sustainability 2024 , 16 , 3872. [ Google Scholar ] [ CrossRef ]
• Islam, Q.; Khan, A.F.M.S. Assessing Consumer Behavior in Sustainable Product Markets: A Structural Equation Modeling Approach with Partial Least Squares Analysis. Sustainability 2024 , 16 , 3400. [ Google Scholar ] [ CrossRef ]
• Dai, A. Research on Intelligent Modeling and Control of Grain Drying Process. Ph.D. Thesis, Beijing University of Posts and Telecommunications, Beijing, China, 2019. [ Google Scholar ]
• Kamruzzaman, M.D.; Uyeh, D.D.; Jang, I.J.; Woo, S.M.; Ha, Y.S. Drying characteristics and milling quality of parboiled Japonica rice under various drying conditions. Eng. Agric. Environ. Food 2017 , 10 , 292–297. [ Google Scholar ] [ CrossRef ]
• Lei, D. Mixed-flow drying tower grain moisture content control model and control system research. Master’s Thesis, Heilongjiang Bayi Agricultural University, Daqing, China, 2023. [ Google Scholar ]

Click here to enlarge figure

Sample Size/Sets	Maximum Value/%	Minimum Value/%	Mean Value/%	SD/%	C.V.
207	23.01	12.41	2.962	2.691	0.153

Names	Ambient	Upper Outlet	Lower Outlet	Drying Section A	Drying Section B	Drying Section C	Drying Section D	Drying Section E	Hot Air Pipe
Pearson correlation coefficient	0.040	0.861	0.809	0.765	0.882	0.781	0.874	0.709	0.360
Significance	0.569	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001

Names	Ambient	Upper Outlet	Lower Outlet	Drying Section A	Drying Section B	Drying Section C	Drying Section D	Drying Section E	Hot Air Pipe
Pearson correlation coefficient	−0.121	−0.749	−0.770	−0.714	−0.752	−0.721	−0.790	−0.816	−0.452
Significance	0.083	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001

Variable Names	Unnormalized Coefficient	Standard Error	Standardization Coefficient	VIF	Significance
Constant	22.596	1.094	-	-	<0.001
Upper outlet temperature	0.139	0.085	0.575	44.91	0.105
Lower outlet temperature	0.029	0.093	0.117	11.320	0.756
Drying section A temperature	0.199	0.099	0.851	36.012	0.046
Drying section B temperature	−0.31	0.058	−1.458	29.79	<0.001
Drying section C temperature	−0.381	0.119	−1.56	74.06	0.002
Drying section D temperature	0.088	0.039	0.106	3.37	0.027
Drying section E temperature	0.09	0.017	0.587	17.73	<0.001
Upper outlet humidity	0.041	0.023	0.389	21.52	0.078
Lower outlet humidity	0.026	0.036	0.178	23.17	0.477
Humidity in drying section A	0.128	0.03	0.591	29.33	<0.001
Humidity in drying section B	−0.01	0.01	−0.041	2.939	<0.001
Humidity in drying section C	−0.093	0.03	−0.438	30.65	0.003
Humidity in drying section D	−0.102	0.033	−0.881	53.16	0.003
Humidity in drying section E	−0.062	0.045	−0.076	4.6	0.172

Model Evaluation Index	R	Adjusted R	Root-Mean-Square Error	Significance
value	0.919	0.910	0.881	<0.001

Variable Names	Unnormalized Coefficient	Standard Error	Standardization Coefficient	VIF	Significance
Constant	22.952	0.718	-	-	<0.001
Upper outlet temperature	0.143	0.069	0.592	8.800	0.041
Lower outlet temperature	0.114	0.010	0.740	2.060	<0.001
Drying section B temperature	−0.348	0.053	−1.636	11.861	<0.001
Drying section C temperature	−0.183	0.089	−0.751	7.270	<0.001
Lower outlet humidity	0.075	0.029	0.090	1.630	<0.001
Humidity in drying section A	0.012	0.026	0.056	3.160	0.742
Humidity in drying section B	−0.057	0.017	−0.535	3.740	<0.001
Humidity in drying section D	0.010	0.025	0.085	4.342	0.701

Model Evaluation Index	R	Adjusted R	Root-Mean-Square Error	Significance
value	0.905	0.898	0.908	<0.001

Batch Size	Initial Learn Rate	Iterations	RMSE (Training Set)/%	RMSE (Validation Set)/%	Training Duration/s
30	0.01	50	0.983	1.022	2
		100	0.865	0.876	3
		500	0.883	0.861	9
		1000	0.852	0.946	18
50	0.01	50	1.002	0.961	1
		100	0.875	0.896	2
		500	0.892	0.905	5
		1000	0.887	0.922	9
100	0.01	50	1.057	1.131	1
		100	1.201	1.242	2
		500	0.892	0.935	5
		1000	0.912	0.960	9

Batch Size	Initial Learn Rate	Iterations	RMSE (Training Set)/%	RMSE (Validation Set)/%	Training Duration/s
30	0.01	50	0.774	0.749	2
		100	0.711	0.697	2
		500	0.690	0.723	9
		1000	0.649	0.793	17
50	0.01	50	0.997	0.833	1
		100	0.755	0.714	2
		500	0.769	0.809	4
		1000	0.731	0.752	9
100	0.01	50	1.057	1.233	1
		100	1.201	1.141	2
		500	0.862	0.915	5
		1000	0.740	0.819	9

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Xing, S.; Lin, Z.; Gao, X.; Wang, D.; Liu, G.; Cao, Y.; Liu, Y. Research on Outgoing Moisture Content Prediction Models of Corn Drying Process Based on Sensitive Variables. Appl. Sci. 2024 , 14 , 5680. https://doi.org/10.3390/app14135680

Xing S, Lin Z, Gao X, Wang D, Liu G, Cao Y, Liu Y. Research on Outgoing Moisture Content Prediction Models of Corn Drying Process Based on Sensitive Variables. Applied Sciences . 2024; 14(13):5680. https://doi.org/10.3390/app14135680

Xing, Simin, Zimu Lin, Xianglan Gao, Dehua Wang, Guohui Liu, Yi Cao, and Yadi Liu. 2024. "Research on Outgoing Moisture Content Prediction Models of Corn Drying Process Based on Sensitive Variables" Applied Sciences 14, no. 13: 5680. https://doi.org/10.3390/app14135680

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals