Distributed Systems and Parallel Computing
No matter how powerful individual computers become, there are still reasons to harness the power of multiple computational units, often spread across large geographic areas. Sometimes this is motivated by the need to collect data from widely dispersed locations (e.g., web pages from servers, or sensors for weather or traffic). Other times it is motivated by the need to perform enormous computations that simply cannot be done by a single CPU.
From our company’s beginning, Google has had to deal with both issues in our pursuit of organizing the world’s information and making it universally accessible and useful. We continue to face many exciting distributed systems and parallel computing challenges in areas such as concurrency control, fault tolerance, algorithmic efficiency, and communication. Some of our research involves answering fundamental theoretical questions, while other researchers and engineers are engaged in the construction of systems to operate at the largest possible scale, thanks to our hybrid research model .
Recent Publications
Some of our teams.
Algorithms & optimization
Graph mining
Network infrastructure
System performance
We're always looking for more talented, passionate people.
IEEE Account
- Change Username/Password
- Update Address
Purchase Details
- Payment Options
- Order History
- View Purchased Documents
Profile Information
- Communications Preferences
- Profession and Education
- Technical Interests
- US & Canada: +1 800 678 4333
- Worldwide: +1 732 981 0060
- Contact & Support
- About IEEE Xplore
- Accessibility
- Terms of Use
- Nondiscrimination Policy
- Privacy & Opting Out of Cookies
A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.
Navigation Menu
Search code, repositories, users, issues, pull requests..., provide feedback.
We read every piece of feedback, and take your input very seriously.
Saved searches
Use saved searches to filter your results more quickly.
To see all available qualifiers, see our documentation .
- Notifications You must be signed in to change notification settings
A curated list to learn about distributed systems
theanalyst/awesome-distributed-systems
Folders and files.
Name | Name | |||
---|---|---|---|---|
128 Commits | ||||
Repository files navigation
Awesome-distributed-systems.
A (hopefully) curated list on awesome material on distributed systems, inspired by other awesome frameworks like awesome-python . Most links will tend to be readings on architecture itself rather than code itself.
Read things here before you start.
- CAP Theorem , Also plain english explanation
- Fallacies of Distributed Computing , expect things to break, everything
- Distributed systems theory for the distributed engineer , most of the papers/books in the blog might reappear in this list again. Still a good BFS approach to distributed systems.
- FLP Impossibility Result (paper) , an easier blog post to follow along
- An Introduction to Distributed Systems @aphyr's excellent introduction to distributed systems
- Distributed Systems for fun and profit [Free]
- Distributed Systems Principles and Paradigms, Andrew Tanenbaum [Free with registration]
- Scalable Web Architecture and Distributed Systems [Free]
- Principles of Distributed Systems [Free] [ETH Zurich University]
- Making reliable distributed systems in the presence of software errors , [Free] Joe Amstrong's (Author of Erlang) PhD thesis
- Designing Data Intensive Applications [Amazon Link]
- Distributed Machine Learning Patterns, Yuan Tang , Practical patterns for scaling machine learning from your laptop to a distributed cluster
- Distributed Computing, Hagit Attiya and Jennifer Welch
- Distributed Algorithms, Nancy Lynch [Amazon Link]
- Impossibility Results for Distributed Computing [paywall]
- Designing Distributed Systems, Brendan Burns [Free with registration]
- Distributed Systems: Concepts and Design, George Coulouris [Amazon Link]
- Akka in Action, Second Edition
- Systemantics: how systems work and especially how they fail
- Think Distributed Systems [Free with subscription]
Must read papers on distributed systems. While nearly all of Lamport's work should feature here, just adding a few that must be read.
- Times, Clocks and Ordering of Events in Distributed Systems Lamport's paper, the Quintessential distributed systems primer
- Session Guarantees for Weakly Consistent Replicated Data a '94 paper that talks about various recommendations for session guarantees for eventually consistent systems, many of this would be standard vocabulary in reading other dist. sys papers, like monotonic reads, read your writes etc.
Storage & Databases
- Dynamo: Amazon's Highly Available Key Value Store Paraphrasing @fogus from their blog , it is very rare for a paper describing an active production system to influence the state of active research in any industry; this is one of those seminal distributed systems paper that solves the problem of a highly available and fault tolerant database in an elegant way, later paving the way for systems like Cassandra, and many other AP systems using a consistent hashing.
- Bigtable: A Distributed Storage System for Structured Data
- The Google File System
- Cassandra: A Decentralized Structured Storage System Inspired heavily by Dynamo, an now an open source
- CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data , the algorithm for the basis of Ceph distributed storage system, for the architecture itself read RADOS
Messaging systems
- The Log: What every software engineer should know about real-time data's unifying abstraction , a somewhat long read, but covers brilliantly on logs, which are at the heart of most distributed systems
- Kafka: a Distributed Messaging System for Log Processing
Distributed Consensus and Fault-Tolerance
- Practical Byzantine Fault Tolerance
- The Byzantine Generals Problem
- Impossibility of Distributed Consensus with One Faulty Process
- The Part Time Parliament Paxos, Lamport's original Paxos paper, a bit difficult to understand, may require multiple passes
- Paxos Made Simple , a more terse readable Paxos paper by Lamport himself. Shorter and more easier compared to the original.
- The Chubby Lock Service for loosely coupled distributed systems Google's lock service used for loosely coupled distributed systems. Sort of Paxos as a Service for building other distributed systems. Primary inspiration behind other Service Discovery & Coordination tools like Zookeeper, etcd, Consul etc.
- Paxos made live - An engineering perspective Google's learning while implementing systems atop of Paxos. Demonstrates various practical issues encountered while implementing a theoretical concept.
- Raft Consensus Algorithm An alternative to Paxos for distributed consensus, that is much simpler to understand. Do checkout an interesting visualization of raft
- Conflict-free Replicated Data Types presents an approach for Strong Eventual Consistency which as been applied in projects such as Riak , Redis and Akka . A great talk on the subject by Martin Kleppmann can be found here
- Speculative algorithms for global state synchronizations Azos.Sky.Server.Locking uses probability based QOS (Quality of Service)/Trust measure to ensure probability-based consensus. The approach avoids distributed state machine/phase synchronization and is very simple to understand and implement
Testing, monitoring and tracing
While designing distributed systems are hard enough, testing them is even harder.
- Dapper , Google's large scale distributed-systems tracing infrastructure, this was also the basis for the design of open source projects such as Zipkin , Apache SkyWalking , Pinpoint and HTrace .
Programming Models
- Distributed Programming Model
- PSync: a partially synchronous language for fault-tolerant distributed algorithms Video: Conference Video
- Programming Models for Distributed Computing
- Logic and Lattices for Distributed Programming
Verification of Distributed Systems
- Curated list of resources on testing distributed systems includes links to materials on testing by various companies (Google, Amazon, Netflix, Microsoft, Dropbox, etc) and research papers.
- Jepsen A framework for distributed systems verification, with fault injection @aphyr has featured enough times in this list already, but Jepsen and the blog posts that go with are a quintessntial addition to any distributed systems reading list.
- Verdi A Framework for Implementing and Formally Verifying Distributed Systems Paper
- Distributed Deep Dive interview series by Ably Relatime .
- Distributed Systems in One Lesson Distributed Systems in One Lesson by Tim Berglund
- Reliable Distributed Algorithms, Part 1 , KTH Sweden
- Reliable Distributed Algorithms, Part 2 , KTH Sweden
- Cloud Computing Concepts , University of Illinois
- CMU: Distributed Systems in Go Programming Language
- Software Defined Networking , Georgia Tech.
- ETH Zurich: Distributed Systems
- ETH Zurich: Distributed Systems Part 2 , covers Distributed control algorithms, communication models, fault-tolerance among other things. In particular fault tolerance issues (models, consensus, agreement) and replication issues (2PC,3PC, Paxos), which are critical in understanding distributed systems are explained in great detail.
- Distributed Systems Course , A beginner course on distributed system by Chris Colohan, A google employee who contributed to SUIF, MapReduce, TCMalloc, Percolator, Caffeine, Borg, Omega, and Piper.
- MIT 6.824 , Youtube-playlist MIT distributed system lectures, in each video they discuss papers like GFS, Zookeeper, RAFT, Spanner...
- Distributed Systems , Lectures 9 to 16 of the Cambridge University lecture "Concurrent and Distributed Systems", given by Dr. Martin Kleppmann. Youtube-playlist . A computer science entrance course, covered basic models and algorithms in distributed systems, also discussed CRDT, collaboration software and google's spanner.
Blogs and other reading links
- Amazon Builder's Library , a collection of Amazon's learnings on distributed systems
- How we implemented consistent hashing efficiently
- Notes on Distributed Systems for Young Bloods
- High Scalability Several architectures of huge internet services, for eg twitter , whatsapp
- There is No Now , Problems with simultaneity in distributed systems
- Turing Lecture: The Computer Science of Concurrency: The Early Years , An article by Leslie Lamport on concurrency
- The Paper Trail blog, a very readable blog covering various aspects of distributed systems
- aphyr , Posts on jepsen series are pretty awesome
- All Things Distributed - Wernel Vogel's (Amazon CTO) blog on distributed systems
- Distributed Systems: Take Responsibility for Failover
- The C10K problem
- On Designing and Deploying Internet-Scale Services
- Files are hard A blog post on filesystem consistency, pretty important to read if you are into distributed storage or databases.
- Distributed Systems Testing: The Lost World Testing distributed systems are hard enough, a well researched blog post which again covers a lot of links to various approaches and other papers
- SWIM Protocol explained A blog post on popular SWIM failure detector
- ACM Symposium on Principles of Distributed Computing (PODC) and International Symposium on Distributed Computing (DISC) , a list of resources from PODC–DISC community including conference series, mailing lists, youtube, twitter, etc.
- IEEE International Parallel & Distributed Processing Symposium (IPDPS) , an international forum for engineers and scientists to present their latest research findings.
- Springer Distributed Computing Journal , a journal about theory, design, specification, and implementation of distributed systems.
Other lists like this one
- Readings in distributed systems
- Distributed Systems meta list
- List of required readings for Distributed Systems Part of CMU's Engineering Distributed Systems course
- The Distributed Reader
- A Distributed Systems Reading List , A collection of material, mostly papers on Distributed Systems Theory as well as seminal industry papers
- Distributed Systems Readings , A comprehensive list of online courses related to distributed systems
- Awesome Distributed Consensus , Another list of materials on distributed consensus protocols
- Beginner's Guide to Distributed Systems A blog post with some useful getting started links for distributed systems
Contributors 31
A brief introduction to distributed systems
- October 2016
- Computing 98(10)
- University of Twente
- Vrije Universiteit Amsterdam
Abstract and Figures
Discover the world's research
- 25+ million members
- 160+ million publication pages
- 2.3+ billion citations
- KNOWL INF SYST
- Marwan Aziz Mohammed
- Husam Barjas Bofaoor
- Abid Faisal Ayon
- S M Maksudul Alam
- Jean Baptiste Minani
- Yahia El Fellah
- Gabriella Carini
- Yanzhu Chen
- Pavel Cenkl
- Mustafa Ghani
- Holger Giese
- Poopak Alaeifar
- Shantanu Pal
- Jason R. Delker
- John M. Everson
- James W. Norris
- Jason K. Whitney
- Carl Kesselman
- Matthew Dobson
- Recruit researchers
- Join for free
- Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up
A Comparative Study of Consensus Algorithms for Distributed Systems
- Conference paper
- First Online: 12 April 2022
- Cite this conference paper
- Kelsi Rado Van Dame 8 ,
- Thomas Bronson Bergmann 8 ,
- Mohamed Aichouri 8 &
- Maria Pantoja ORCID: orcid.org/0000-0002-1942-9769 8
Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1540))
Included in the following conference series:
- Latin American High Performance Computing Conference
471 Accesses
1 Citations
Distributed Systems (DS) where multiple computers share a workload across a network, are used everywhere, from data intensive computations to storage and machine learning. DS provide a relatively cheap and efficient solution that allows stability with improved performance for computational intensive applications. Fundamental to DS is the consensus algorithm, necessary to agree on which server is the master, who has a lock and many other applications. Consensus algorithms are sometimes very difficult to understand and therefore implement correctly. In this paper we chose to complete a comparative study between three different consensus algorithms Raft, Paxos, and pBFT. We provided our implementation for the three algorithms with details of the assumptions taken. The goal of this study is to better understand the differences between the systems in terms of performance and assess the advantages and disadvantages of each. To test the performance of each program, we recorded consensus latency vs. node count and we present a summary of our results in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Subscribe and save.
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
- Available as PDF
- Read on any device
- Instant download
- Own it forever
- Available as EPUB and PDF
- Compact, lightweight edition
- Dispatched in 3 to 5 business days
- Free shipping worldwide - see info
Tax calculation will be finalised at checkout
Purchases are for personal use only
Institutional subscriptions
Similar content being viewed by others
Automatic Deployment of a Consensus Networks MAS
Distributed Computing
On Performance Evaluation of Distributed System Size Estimation Executed by Average Consensus Weights
Castro, M., Liskov, B.: Practical Byzantine Fault Tolerance. Massachusetts Institute of Technology (2019). http://pmg.csail.mit.edu/papers/osdi99.pdf 6 Dec 2020
Seeley, L.: Introduction to Sawtooth PBFT, Hyperledger (2019). https://www.hyperledger.org/blog/2019/02/13/introduction-to-sawtooth-pbft 6 Dec 2020
Howard, H., Mortier, R.: Paxos vs Raft: have we reached consensus on distributed consensus? University of Cambridge (2020). https://arxiv.org/pdf/2004.05074.pdf 2 Dec 2020
Lamport, L.: Paxos Made Simple, ACM SIGACT News (2001). https://lamport.azurewebsites.net/pubs/paxos-simple.pdf 3 Dec 2020
Lamport, L.: The part-time parliament. ACM Trans. Comput. Syst. 16 (2), 133–169 (1998)
Article Google Scholar
Lamport, L.: Fast paxos. Distrib. Comput. 19 (2), 79–103 (2006)
Article MathSciNet Google Scholar
Lampson, B.W.: The ABCD’s of paxos. In: Proceedings of PODC 2001, ACM Symposium on Principles of Distributed Computing, p. 13. ACM (2001)
Google Scholar
GXUI - A Go cross platform UI library (2015). https://github.com/google/gxui
Ongaro, D., Ousterhout, J.: In search of an understandable consensus algorithm. In: Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference (USENIX ATC 2014), pp. 305–320. USENIX Association, USA (2014)
Bolosky, W.J., Bradshaw, D., Haagens, R.B., Kusters, N.P., Li, P.: Paxos replicated state machines as the basis of a high-performance data store. In: Proceedings of NSDI 2011, USENIX Conference on Networked Systems Design and Implementation, pp. 141–154. USENIX (2011)
Chandra, T.D., Griesemer, R., Redstone, J.: Paxos made live: an engineering perspective. In: Proceedings of PODC 2007, ACM Symposium on Principles of Distributed Computing, pp. 398–407. ACM (2007)
Hunt, P., Konar, M., Junqueira, F.P., Reed, B.: ZooKeeper: wait-free coordination for internet-scale systems. In: Proceedings of ATC 2010, USENIX Annual Technical Conference, pp. 145–158. USENIX (2010)
“Code Repository for the article” Paxos. https://github.com/tbbergmann/569Paxos Raft. https://github.com/theAichouri/Raft569 pBFT. https://github.com/mycalpoly/Practical-Byzantine-Fault-Tolerance-PBFT
Towns, J., et al.: XSEDE: accelerating scientific discovery. Comput. Sci. Eng. 16 (5), 62–74 (2014). https://doi.org/10.1109/MCSE.2014.80
Nystrom, N.A., Levine, M.J., Roskies, R.Z., Scott, J.R.: Bridges: a uniquely flexible HPC resource for new communities and data analytics. In: Proceedings of the 2015 Annual Conference on Extreme Science and Engineering Discovery Environment (St. Louis, MO, 26–30 July 2015). XSEDE15. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2792745.2792775
Download references
Author information
Authors and affiliations.
California Polytechnic State University, San Luis Obispo, CA, 95116, USA
Kelsi Rado Van Dame, Thomas Bronson Bergmann, Mohamed Aichouri & Maria Pantoja
You can also search for this author in PubMed Google Scholar
Corresponding author
Correspondence to Maria Pantoja .
Editor information
Editors and affiliations.
Centro de Investigación y de Estudios Avanzados, Mexico City, Mexico
Isidoro Gitler
Universidad Industrial de Santander, Bucaramanga, Colombia
Carlos Jaime Barrios Hernández
Centro Nacional de Alta Tecnología, San José, Costa Rica
Esteban Meneses
Rights and permissions
Reprints and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper.
Van Dame, K.R., Bergmann, T.B., Aichouri, M., Pantoja, M. (2022). A Comparative Study of Consensus Algorithms for Distributed Systems. In: Gitler, I., Barrios Hernández, C.J., Meneses, E. (eds) High Performance Computing. CARLA 2021. Communications in Computer and Information Science, vol 1540. Springer, Cham. https://doi.org/10.1007/978-3-031-04209-6_9
Download citation
DOI : https://doi.org/10.1007/978-3-031-04209-6_9
Published : 12 April 2022
Publisher Name : Springer, Cham
Print ISBN : 978-3-031-04208-9
Online ISBN : 978-3-031-04209-6
eBook Packages : Computer Science Computer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
- Publish with us
Policies and ethics
- Find a journal
- Track your research
Information
- Author Services
Initiatives
You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.
All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.
Original Submission Date Received: .
- Active Journals
- Find a Journal
- Proceedings Series
- For Authors
- For Reviewers
- For Editors
- For Librarians
- For Publishers
- For Societies
- For Conference Organizers
- Open Access Policy
- Institutional Open Access Program
- Special Issues Guidelines
- Editorial Process
- Research and Publication Ethics
- Article Processing Charges
- Testimonials
- Preprints.org
- SciProfiles
- Encyclopedia
Article Menu
- Subscribe SciFeed
- Recommended Articles
- Google Scholar
- on Google Scholar
- Table of Contents
Find support for a specific problem in the support section of our website.
Please let us know what you think of our products and services.
Visit our dedicated information section to learn more about MDPI.
JSmol Viewer
Innovative load forecasting models and intelligent control strategy for enhancing distributed load levelling techniques in resilient smart grids.
1. Introduction
2. related work, 3. materials and methods, 3.1. dataset collection, 3.1.1. data source, 3.1.2. feature descriptions, 3.2. data pre-processing, 3.2.1. data normalization, 3.2.2. min–max scaling, 3.3. proposed model, 3.3.1. model architectures, 3.3.2. predicting dynamic loads, 3.3.3. intelligent control strategy for load levelling, 3.4. mathematical model, 3.4.1. mathematical model for gated recurrent unit (gru).
- Mathematical Model of LSTM
3.4.2. Optimization Model
3.5. intelligent control strategy, 3.6. evaluation metrics, 3.6.1. mean squared error, 3.6.2. mean absolute percentage error, 4. results and discussion, 4.1. performance of lstm, 4.2. performance of gru, 4.3. impact of intelligent control strategy, 5. conclusions, author contributions, data availability statement, conflicts of interest.
- Ahmad, N.; Ghadi, Y.; Adnan, M.; Ali, M. Load Forecasting Techniques for Power System: Research Challenges and Survey. IEEE Access 2022 , 10 , 71054–71090. [ Google Scholar ] [ CrossRef ]
- Abdelaziz, A.Y.; Biswal, M. Load Forecasting Models in Smart Grid. Encyclopedia. Available online: https://encyclopedia.pub/entry/41526 (accessed on 27 July 2024).
- Khan, A.R.; Mahmood, A.; Safdar, A.; Khan, Z.A.; Khan, N.A. Load forecasting, dynamic pricing and DSM in smart grid: A review. Renew. Sustain. Energy Rev. 2016 , 54 , 1311–1322. [ Google Scholar ] [ CrossRef ]
- Kuster, C.; Rezgui, Y.; Mourshed, M. Electrical load forecasting models: A critical systematic review. Sustain. Cities Soc. 2017 , 35 , 257–270. [ Google Scholar ] [ CrossRef ]
- Salehimehr, S.; Taheri, B.; Sedighizadeh, M. Short-term load forecasting in smart grids using artificial intelligence methods: A survey. J. Eng. 2022 , 2022 , 1133–1142. [ Google Scholar ] [ CrossRef ]
- Wang, X.; Zhang, M.; Ren, F. Load forecasting in a smart grid through customer behaviour learning using L1-regularized continuous conditional random fields. In Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, Singapore, 9–13 May 2016; pp. 817–825. [ Google Scholar ]
- Ferreira, A.; Leitão, P.; Barata, J. Prediction models for short-term load and production forecasting in smart electrical grids. In Proceedings of the Industrial Applications of Holonic and Multi-Agent Systems: 8th International Conference, HoloMAS 2017, Lyon, France, 28–30 August 2017; Proceedings 8. Springer: Berlin/Heidelberg, Germany, 2017; Volume 10444, pp. 186–199. [ Google Scholar ] [ CrossRef ]
- Ali, S. Short Term Load Forecasting using Smart Meter Data. In Proceedings of the Tenth ACM International Conference on Future Energy Systems, Phoenix, AZ, USA, 25–28 June 2019; pp. 419–421. [ Google Scholar ]
- Zheng, J.; Xu, C.; Zhang, Z.; Li, X. Electric load forecasting in smart grids using Long-Short-Term-Memory based Recurrent Neural Network. In Proceedings of the 2017 51st Annual Conference on Information Sciences and Systems (CISS), Baltimore, MD, USA, 22–24 March 2017. [ Google Scholar ] [ CrossRef ]
- Roy, K.; Ishmam, A.; Taher, K.A. Demand forecasting in smart grid using long short-term memory. In Proceedings of the 2021 International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI), Rajshahi, Bangladesh, 8–9 July 2021; Volume 2021, pp. 8–9. [ Google Scholar ] [ CrossRef ]
- Luo, D.S.; Wang, X.K. Research on the framework of smart grid load forecasting platform based on cloud computing. Hunan Daxue Xuebao/J. Hunan Univ. Nat. Sci. 2016 , 43 , 101–108. [ Google Scholar ]
- Islam, B.U.; Rasheed, M.; Ahmed, S.F. Review of Short-Term Load Forecasting for Smart Grids Using Deep Neural Networks and Metaheuristic Methods. Math. Probl. Eng. 2022 , 2022 , 4049685. [ Google Scholar ] [ CrossRef ]
- Dhumale, S. Smart Grid experimentation Plan for Load Forecasting—A Case study of a large power distribution company. Proc. Int. Jt. J. Conf. Eng. Technol. 2010 , 3 , 78–83. [ Google Scholar ]
- Chemetova, S.; Santos, P.; Ventim-Neves, M. Load forecasting in electrical distribution grid of medium voltage. IFIP Adv. Inf. Commun. Technol. 2016 , 470 , 340–349. [ Google Scholar ] [ CrossRef ]
- Lee, D.; Park, J.; Kundur, D. Trends in short-term renewable and load forecasting for applications in smart grid. Lect. Notes Inst. Comput. Sci. Soc. Telecommun. Eng. 2016 , 166 , 292–300. [ Google Scholar ] [ CrossRef ]
- Ibrahim, B.; Rabelo, L.; Gutierrez-Franco, E.; Clavijo-Buritica, N. Machine Learning for Short-Term Load Forecasting in Smart Grids. Energies 2022 , 15 , 8079. [ Google Scholar ] [ CrossRef ]
- Habbak, H.; Mahmoud, M.; Metwally, K.; Mohamed, I. Smart Load Forecasting Techniques and Their Applications in Smart Grids Hany. Green Energy Technol. 2012 , 101 , 119–121. [ Google Scholar ]
- Zhang, D.; Jin, X.; Shi, P.; Chew, X.Y. Real-time load forecasting model for the smart grid using bayesian optimized CNN-BiLSTM. Front. Energy Res. 2023 , 11 , 1193662. [ Google Scholar ] [ CrossRef ]
- Vazquez, R.; Amaris, H.; Alonso, M.; Lopez, G.; Moreno, J.I.; Olmeda, D.; Coca, J. Assessment of an Adaptive Load Forecasting Methodology in a Smart Grid Demonstration Project. Energies 2017 , 10 , 190. [ Google Scholar ] [ CrossRef ]
- Ann, O.N.; Forest, R. Short Term Electricity Load Forecasting for Tumakuru City Based Short Term Electricity Load Forecasting for Tumakuru City Based on Ann, Lstm and Random Forest. Int. J. Creat. Res. Thoughts 2022 , 10 , 2023. [ Google Scholar ]
- Fotopoulou, M.; Pediaditis, P.; Skopetou, N.; Rakopoulos, D.; Christopoulos, S.; Kartalidis, A. A Review of the Energy Storage Systems of Non-Interconnected European Islands. Sustainability 2024 , 16 , 1572. [ Google Scholar ] [ CrossRef ]
- Esnaola-Gonzalez, I.; Jelić, M.; Pujić, D.; Diez, F.J.; Tomašević, N. An AI-Powered System for Residential Demand Response. Electronics 2021 , 10 , 693. [ Google Scholar ] [ CrossRef ]
- Rai, S.; De, M. Load forecasting using Two-level Heterogeneous Ensemble Method for Smart Metered Distribution System. Sci. Iran. 2023 . [ Google Scholar ] [ CrossRef ]
- Azeem, A.; Ismail, I.; Jameel, S.M.; Romlie, F.; Danyaro, K.U.; Shukla, S. Deterioration of Electrical Load Forecasting Models in a Smart Grid Environment. Sensors 2022 , 22 , 4363. [ Google Scholar ] [ CrossRef ]
- Madrid, E.A.; Antonio, N. Short-term electricity load forecasting with machine learning. Information 2021 , 12 , 50. [ Google Scholar ] [ CrossRef ]
- Patro, S.G.K.; Sahu, K.K. Normalization: A preprocessing stage. arXiv 2015 , arXiv:1503.06462. [ Google Scholar ] [ CrossRef ]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997 , 9 , 1735–1780. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv 2014 , arXiv:1406.1078.
- Willmott, C.T.; Matsuura, K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 2005 , 30 , 79–82. [ Google Scholar ] [ CrossRef ]
- Evaluating accuracy (or error) measures. Am. Stat. 1995 , 49 , 290–293. [ CrossRef ]
Click here to enlarge figure
Feature | Description |
---|---|
Date and time | The identifier of the time and date that data were collected. |
Temperature (°C) | The temperature is in degrees Celsius at the given timestamp. Temperature can significantly impact energy usage. |
Load (MW) | The energy consumption in megawatts (MW) at the corresponding timestamp and location. |
Price (Cents/kWh) | The cost of energy in cents per kilowatt-hour (kWh) at the given time and location. |
Test Dataset | LSTM Model |
---|---|
AEP_hourly.csv | 162.435 |
COMED_hourly.csv | 58.772 |
DAYTON_hourly.csv | 29.821 |
DEOK_hourly.csv | 93.464 |
DOM_hourly.csv | 23.962 |
Test Dataset | LSTM Model |
---|---|
AEP_hourly.csv | 0.546% |
COMED_hourly.csv | 0.667% |
DAYTON_hourly.csv | 0.459% |
DEOK_hourly.csv | 0.621% |
DOM_hourly.csv | 0.418% |
Test Dataset | GRU Model |
---|---|
AEP_hourly.csv | 138.292 |
COMED_hourly.csv | 49.846 |
DAYTON_hourly.csv | 26.612 |
DEOK_hourly.csv | 79.110 |
DOM_hourly.csv | 22.988 |
Test Dataset | GRU Model |
---|---|
AEP_hourly.csv | 0.501% |
COMED_hourly.csv | 0.618% |
DAYTON_hourly.csv | 0.401% |
DEOK_hourly.csv | 0.568% |
DOM_hourly.csv | 0.391% |
Test Dataset | LSTM Model MSE | GRU Model MSE | LSTM Model MAPE | GRU Model MAPE |
---|---|---|---|---|
AEP_hourly.csv | 162.435 | 138.292 | 0.546% | 0.501% |
COMED_hourly.csv | 58.772 | 49.846 | 0.667% | 0.618% |
DAYTON_hourly.csv | 29.821 | 26.612 | 0.459% | 0.401% |
DEOK_hourly.csv | 93.464 | 79.110 | 0.621% | 0.568% |
DOM_hourly.csv | 23.962 | 22.988 | 0.418% | 0.391% |
Reference | Technique | Outcome | Limitation |
---|---|---|---|
[ ] | AMI data | Increased forecasting accuracy, privacy, and data quality concerns | Computational complexity |
[ ] | Deep neural networks with metaheuristic approaches | Improved accuracy of short-term load estimates | Evaluation of method impact on grid performance |
[ ] | LSTM-based recurrent neural networks | Enhancing the accuracy of electric load forecasting | - |
[ ] | Medium-voltage distribution networks | Procedure for estimating energy consumption | Tailored algorithms for infrastructure needed |
[ ] | Cognitive algorithms | Development of load forecasting methods for smart grids | Vital for efficient grid management and reliability |
[ ] | Smart meter data-driven algorithms | Comparison of load forecasting approaches utilizing smart meter data | Enhancing the accuracy of load forecasts, guiding effective grid management |
[ ] | Adaptive load forecasting technique | Examination of adaptive forecasting methods for smart grids | Insights into possibilities and practicality of such methods |
[ ] | Cloud computing | Improved accuracy in load forecasting | Dependency on cloud infrastructure |
Proposed | LSTM-GRU | Commendable predictive capabilities demonstrated by low MSE and MAPE values | The scope may not fully represent global energy consumption patterns despite utilizing diverse datasets |
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
Share and Cite
Fangzong, W.; Nishtar, Z. Innovative Load Forecasting Models and Intelligent Control Strategy for Enhancing Distributed Load Levelling Techniques in Resilient Smart Grids. Electronics 2024 , 13 , 3552. https://doi.org/10.3390/electronics13173552
Fangzong W, Nishtar Z. Innovative Load Forecasting Models and Intelligent Control Strategy for Enhancing Distributed Load Levelling Techniques in Resilient Smart Grids. Electronics . 2024; 13(17):3552. https://doi.org/10.3390/electronics13173552
Fangzong, Wang, and Zuhaib Nishtar. 2024. "Innovative Load Forecasting Models and Intelligent Control Strategy for Enhancing Distributed Load Levelling Techniques in Resilient Smart Grids" Electronics 13, no. 17: 3552. https://doi.org/10.3390/electronics13173552
Article Metrics
Article access statistics, further information, mdpi initiatives, follow mdpi.
Subscribe to receive issue release notifications and newsletters from MDPI journals
IMAGES
VIDEO
COMMENTS
Distributed systems have been an active field of research for over 60 years, and has played a crucial role in computer science, enabling the invention of the Internet that underpins all facets of modern life. Through technological advancements and their changing role in society, distributed systems have undergone a perpetual evolution, with each change resulting in the formation of a new ...
Distributed computing systems refer to a network of computers that work together to. achieve a common goal. In a distributed computing system, individual computers are. connected to each other ...
D istributed computing refers to two or more. computers networked together sharing the same. computing work. The objective of distributed co mputing is. to sharing the job between multiple ...
The publishes original research papers and timely review articles on the theory, design, evaluation, and use of parallel and/or distributed computing systems. The journal also features special issues on these topics; again covering the full range from the design to the use of our targeted systems. Research Areas Include:
Overview. Distributed Computing is a peer-reviewed journal that serves as a forum for significant contributions to the theory and practical aspects of distributed systems. Covers topics from design and analysis of distributed algorithms to architectures and protocols for communication networks. Includes discussions on synchronization protocols ...
From our company's beginning, Google has had to deal with both issues in our pursuit of organizing the world's information and making it universally accessible and useful. We continue to face many exciting distributed systems and parallel computing challenges in areas such as concurrency control, fault tolerance, algorithmic efficiency, and ...
Explore the latest full-text research PDFs, articles, conference papers, preprints and more on DISTRIBUTED COMPUTING. Find methods information, sources, references or conduct a literature review ...
The distributed computing system is a hot research field. Deep learning, the Internet of Things, and other technologies are rapidly advancing, necessitating better levels of computation, storage, and communication efficiency for many jobs and applications. The advanced distributed computing model, represented by cloud computing and edge computing, has attracted the attention of researchers ...
Abstract. In this paper, we review the background and the state of the art of the Distributed Computing software stack. We aim to provide the readers with a comprehensive overview of this area by supplying a detailed big-picture of the latest technologies. First, we introduce the general background of Distributed Computing and propose a layered ...
The early stand-alone mode has not met people's needs, and the network-based collaborative distributed gained more and more attention and favor. Distributed computing system is one of the hottest Internet research directions in the era of big data era. It has the characteristics of high efficiency, high capacity, dynamic processing, and so on.
Distributed computer systems have been the subject of a vast amount of research. Many prototype distributed computer systems have been built at university, industrial, commercial, and government research laboratories, and production systems of all sizes and types have proliferated. It is impossible to survey all distributed computing system research. Instead, this paper identifies six ...
By an extensive literature review (academic research papers and whitepapers written by different cloud service providers, hardware and software vendors [45-51] ... Distributed computing systems are usually treated differently from parallel computing systems or shared-memory systems, where multiple computers share a common memory pool that is ...
There are classes of computing devices working together in DCCSs to enable seamless data generation, processing, analysis, and communication across the system. In our paper, we categorize these devices into five groups: embedded computers, IoT, mobile devices, desktop computers, and servers. 3.1.1. Embedded Computers.
Distributed computing is a field of computer science that studies distributed systems. ... Explore the latest full-text research PDFs, articles, conference papers, preprints and more on ...
scalable, energy-efficient, reliable, verifiable, and secure computing systems. 2 Recommendations This section briefly summarizes top-level recommendations for research in parallel and distributed computing. Subsequent sections of the report provide more detail on specific research directions.
Distributed systems are by now commonplace, yet remain an often difficult area of research. This is partly explained by the many facets of such systems and the inherent difficulty to isolate these facets from each other. In this paper we provide a brief overview of distributed systems: what they are, their general design goals, and some of the most common types.
This article discusses four fundamental topics for future Distributed Computing Continuum Systems: their representation, model, lifelong learning, and business model. Further, it presents techniques and concepts that can be useful to define these four topics specifically for Distributed Computing Continuum Systems. Finally, this article presents a broad view of the synergies among the ...
In this paper we review several aspects of the scheduling literature for distributed systems, and proposed a taxonomy that encompasses the scheduler organization in the system, the scheduler input and output data, the frequency the scheduler runs, the application model and target system, and the scheduling objectives.
Must read papers on distributed systems. While nearly all of Lamport's work should feature here, just adding a few that must be read.. Times, Clocks and Ordering of Events in Distributed Systems Lamport's paper, the Quintessential distributed systems primer; Session Guarantees for Weakly Consistent Replicated Data a '94 paper that talks about various recommendations for session guarantees for ...
Written by a highly qualified academic with significant research experience in the field, Resource Management on Distributed Systems includes information on sample topics such as: Attributes of parallel/distributed applications that have an intimate relationship with system behavior and performance, plus their related performance metrics ...
Distributed backup placement. Leonid Barenboim. Gal Oren. OriginalPaper 24 March 2022 Pages: 455 - 473. 1. …. Next. Distributed Computing is a peer-reviewed journal that serves as a forum for significant contributions to the theory and practical aspects of distributed ...
1. Distributed computer control The main aim of this paper is to present a review of the literature concerning distributed systems and large scale systems modelling and conjunction of these subjects. In the experimental part, an example of a large scale system model design is presented.
This review paper thoroughly explores the impact of artificial intelligence on the planning and operation of distributed energy systems in smart grids. With the rapid advancement of artificial intelligence techniques such as machine learning, optimization, and cognitive computing, new opportunities are emerging to enhance the efficiency and reliability of electrical grids.
A distributed system is a collection of autonomous computing elements that. appears to its users as a single coherent system. This definition refers to two characteristic features of distributed ...
For this reason, in this paper we sought to compare differences in consensus latency vs node count in each of 3 algorithms: Raft [9], Paxos [4], and pBFT [1]. There are three main properties in distributed consensus algorithms; fault tolerance, liveness, and safety. (a) Fault Tolerance.
Dynamic load forecasting is essential for effective energy management and grid operation. The use of GRU (Gated Recurrent Unit) and Long Short-Term Memory (LSTM) networks for precise load prediction is investigated in this paper. This research examines dynamic load patterns by innovatively integrating heterogeneous information from several datasets. The results show that the LSTM and GRU ...