Energy Aware Data Centers and Networks: a Survey
DOI:
https://doi.org/10.26636/jtit.2018.129818Keywords:
energy efficiency, green networks, resource allocation, HPCAbstract
The past years have brought about a great variety of clusters and clouds. This, combined with their increasing size and complexity, has resulted in an obvious need for power-saving control mechanisms. Upon presenting a basis on which such solutions - namely low-level power control interfaces, CPU governors and network topologies – are constructed, the paper summarizes network and cluster resources control algorithms. Finally, the need for integrated, hierarchical control is expressed, and specific examples are provided.
Downloads
References
[1] S. Nedevschi, L. Popa, G. Iannacone, D. Wetherall, and S. Ratnasamy, “Reducing network energy consumption via sleeping and rate adaptation”, in Proc. 5th USENIX Symp. on Netw. Syst. Design and Implement. NSDI 2008, San Francisco, CA, USA, 2008, pp. 323–336.
View in Google Scholar
[2] J. Chabarek et al., “Power awareness in network design and routing”, in Proc. 27th Conf. on Comp. Commun. INFOCOM 2008, Phoenix, AZ, USA, 2008, pp. 457–465. DOI: https://doi.org/10.1109/INFOCOM.2008.93
View in Google Scholar
[3] A. Shehabi et al., “United States data center energy usage report”, 06/2016, 2016 [Online]. Available: https://eta.lbl.gov/publications/ united-states-data-center-energy
View in Google Scholar
[4] R. Bolla, R. Bruschi, A. Carrega, and F. Davoli, “Theoretical and technological limitations of power scaling in network devices”, in Proc. Australasian Telecommun. Netw. and Appl. Conf. ATNAC 2010, Auckland, New Zealand, 2010, pp. 37–42. DOI: https://doi.org/10.1109/ATNAC.2010.5680253
View in Google Scholar
[5] “Advanced Configuration and Power Interface Specification, Revision 5.0”, Hewlett-Packard, Intel, Microsoft, Phoenix Technologies, and Toshiba, 2011 [Online]. Available: http://www.acpi.info/ DOWNLOADS/ACPIspec50.pdf
View in Google Scholar
[6] “IEEE 802.3az Energy Efficient Ethernet Task Force”, IEEE, 2012 [Online]. Available: http://grouper.ieee.org/groups/802/3/az/ public/index.html
View in Google Scholar
[7] M. Karpowicz, “Energy-efficient CPU frequency control for the Linux system”, Concur. and Comput.: Pract. and Exper., vol. 28, no. 2, pp. 420–437, 2015. DOI: https://doi.org/10.1002/cpe.3476
View in Google Scholar
[8] M. Karpowicz, P. Arabas, and E. Niewiadomska-Szynkiewicz, “Design and implementation of energy-aware application-specific CPU frequency governors for the heterogeneous distributed computing systems”, Future Gener. Comp. Syst., vol. 78, pp. 302–315, 2018. DOI: https://doi.org/10.1016/j.future.2016.05.011
View in Google Scholar
[9] “Power saving features in Mellanox products”, Melanox Technologies, 2013.
View in Google Scholar
[10] “Intel 64 and IA-32 Architectures Software Developer’s Manual”, 2015.
View in Google Scholar
[11] S. Terpstra, H. Jagode, H. You, and J. Dongarra, “Collecting performance data with PAPI-C”, in Tools for High Performance Computing 2009. Proceedings of the 3rd International Workshop on Parallel Tools for High Performance Computing, September 2009, ZIH, Dresden, M. S. M¨uller, M. M. Resch, A. Schulz, and W. E. Nagel, Eds. Berlin, Heidelberg: Springer, 2010, pp. 157–173. DOI: https://doi.org/10.1007/978-3-642-11261-4_11
View in Google Scholar
[12] “IPMI – Intelligent Platform Management Interface Specification, Second Generation”, Intel, Hewlett-Packard, NEC, and Dell, 2015.
View in Google Scholar
[13] R. Bolla et al., “The green abstraction layer: A standard power management interface for next-generation network devices”, IEEE Internet Comput., vol. 17, no. 2, pp. 82–86, 2013. DOI: https://doi.org/10.1109/MIC.2013.39
View in Google Scholar
[14] R. Bolla et al., “Large-scale validation and benchmarking of a network of power-conservative systems using ETSI’s green abstraction layer”, Trans. on Emerg. Telecommun. Technol., vol. 27, no. 3, pp. 451–468, 2016. DOI: https://doi.org/10.1002/ett.3006
View in Google Scholar
[15] V. Pallipadi and A. Starikovskiy, “The on-demand governor: past, present and future”, in Proc. Linux Symp., Ottawa, Ontario, Canada, 2006, vol. 2, pp. 223–238.
View in Google Scholar
[16] K. Accardi, “Balancing power and performance in the Linux kernel”, 2015.
View in Google Scholar
[17] P. Arabas and M. Karpowicz, “Server power consumption: measurements and modeling with measurements”, in Challenges in Automation, Robotics and Measurement Techniques. Proceedings of AUTOMATION-2016, March 2-4, 2016, Warsaw, Poland, R. Szewczyk, C. Zieliński, and M. Kaliczyńska, Eds. Springer, 2016, pp. 233–244 (ISBN 9783319293578). DOI: https://doi.org/10.1007/978-3-319-29357-8_21
View in Google Scholar
[18] J. Gong and Ch. Xu, “A gray-box feedback control approach for system-level peak power management”, in Proc. 39th Int. Conf. on Parallel Process. ICPP-2010, San Diego, CA, USA, 2010, pp. 555–564. DOI: https://doi.org/10.1109/ICPP.2010.63
View in Google Scholar
[19] R. Bolla, R. Bruschi, and A. Ranieri, “Green support for PC-based software router: Performance evaluation and modeling”, in IEEE Int. Conf. on Commun., Dresden, Germany, 2009, pp. 1–6. DOI: https://doi.org/10.1109/ICC.2009.5199050
View in Google Scholar
[20] R. Bolla and R. Bruschi, “Energy-aware load balancing for parallel packet processing engines”, in Online Conf. on Green Commun. GreenCom 2011, New York, NY, USA, 2011, pp. 105–112. DOI: https://doi.org/10.1109/GreenCom.2011.6082516
View in Google Scholar
[21] M. Benito, E. Vallejo, and R. Beivide, “On the use of commodity Ethernet technology in exascale HPC systems”, in Proc. IEEE 22nd Int. Conf. on High Perform. Comput. HiPC 2015, Bangalore, India, 2015, pp. 254–263. DOI: https://doi.org/10.1109/HiPC.2015.32
View in Google Scholar
[22] M. Al-Fares, A. Loukissas, and A. Vahdat, “A scalable, commodity data center network architecture”, in Proc. ACM SIGCOMM 2008 Conf. on Data Commun., Seattle, WA, USA, 2008, pp. 63–74. DOI: https://doi.org/10.1145/1402958.1402967
View in Google Scholar
[23] “Cisco Data Center Infrastructure 2.5 Design Guide”, Cisco Systems, Inc., 2011.
View in Google Scholar
[24] C. Clos, “A study of non-blocking switching networks”, The Bell Syst. Tech. J., vol. 32, no. 2, pp. 406–424, 1953. DOI: https://doi.org/10.1002/j.1538-7305.1953.tb01433.x
View in Google Scholar
[25] Ch. E. Leiserson, “Fat-trees: Universal networks for hardwareefficient supercomputing”, IEEE Trans. Comput., vol. 34, no. 10, pp. 892–901, 1985. DOI: https://doi.org/10.1109/TC.1985.6312192
View in Google Scholar
[26] “Introduction to cloud design four design principals for IaaS”, Melanox Technologies, 2012.
View in Google Scholar
[27] Y. Xia and T. S. E. Ng, “Flat-tree: A convertible data center network architecture from CLOS to random graph”, in Proc. 15th ACM Worksh. on Hot Topics in Netw. HotNets’16, Atlanta, GA, USA, 2016, pp. 71–77. DOI: https://doi.org/10.1145/3005745.3005763
View in Google Scholar
[28] J. Kim, W. J. Dally, and D. Abts, “Flattened butterfly: A cost-efficient topology for high-radix networks”, in Proc. of the 34th Ann. Int. Symp. on Comp. Architec. ISCA’07, San Diego, CA, USA, 2007, pp. 126–137. DOI: https://doi.org/10.1145/1250662.1250679
View in Google Scholar
[29] A. Shpiner, Z. Haramaty, S. Eliad, V. Zdornov, B. Gafni, and E. Zahavi, “Dragonfly+: Low cost topology for scaling datacenters”, in Proc. IEEE 3rd Int. Worksh. on High-Perform. Interconnec. Netw. in the Exascale and Big-Data Era HiPINEB 2017, 2017, Austin, TX, USA, pp. 1–8. DOI: https://doi.org/10.1109/HiPINEB.2017.11
View in Google Scholar
[30] M. Pióro, M. Mysłek, A. Juttner, J. Harmatos, and A. Szentesi, “Topological Design of MPLS Networks”, in Proc. Global Telecommun. Conf. GLOBECOM’2001, San Antonio, TX, USA, 2001.
View in Google Scholar
[31] F. Bianco, G. Cucchietti, and G. Griffa, “Energy consumption trends in the next generation access network – a telco perspective”, in Proc. 29th Inter. Telecommun. Energy Conf. INTELEC 2007), Rome, Italy, 2007, pp. 737–742. DOI: https://doi.org/10.1109/INTLEC.2007.4448879
View in Google Scholar
[32] S. N. Roy, “Energy logic: a road map to reducing energy consumption in telecommunications networks”, in Proc. 30th Int. Telecommun. Energy Conf. INTELEC 2008, San Diego, CA, USA, 2008. DOI: https://doi.org/10.1109/INTLEC.2008.4664025
View in Google Scholar
[33] A. Karbowski and P. Jaskóła, “Two approaches to dynamic power management in energy-aware computer networks – methodological considerations”, in Proc. Federated Conf. on Comp. Sci. and Inform. Syst. FedCSIS, Łódź, Poland, 2015. DOI: https://doi.org/10.15439/2015F228
View in Google Scholar
[34] P. Jaskóła and K. Malinowski, “Two methods of optimal bandwidth allocation in TCP/IP networks with QoS differentiation”, in Proc. Symp. on Perform. Eva. of Comp. and Telecommun. Systems SPECTS’04, San Jose, CA, USA, 2004, pp. 373–378, 2004.
View in Google Scholar
[35] A. Kozakiewicz and K. Malinowski, “Network traffic routing using effective bandwidth theory”, Eur. Trans. on Telecommun., vol. 20, no. 7, pp. 660–667, 2009. DOI: https://doi.org/10.1002/ett.1383
View in Google Scholar
[36] M. Kamola and P. Arabas, “Dynamically established transmission paths in the future internet – proposal of a framework”, Bull. of the Polish Acad. of Sciences: Tech. Sciences, vol. 59, no. 3, pp. 357–366, 2011. DOI: https://doi.org/10.2478/v10175-011-0043-9
View in Google Scholar
[37] K. Malinowski, E. Niewiadomska-Szynkiewicz, and P. Jaskóła, “Price method and network congestion control”, J. of Telecommun. and Inform. Technol., no. 2, pp. 73–77, 2010. DOI: https://doi.org/10.26636/jtit.2010.2.1075
View in Google Scholar
[38] L. Chiaraviglio, M. Mellia, and F. Neri, “Minimizing ISP network energy cost: formulation and solutions”, IEEE/ACM Trans. on Netw., vol. 20, no. 2, pp. 463–476, 2011. DOI: https://doi.org/10.1109/TNET.2011.2161487
View in Google Scholar
[39] E. Niewiadomska-Szynkiewicz, A. Sikora, P. Arabas, and J. Kołodziej, “Control framework for high performance energy aware backbone network”, in Proc. of 26th Eur. Conf. on Modell. and Simul. ECMS 2012, Koblenz, Germany, 2012, pp. 490–496. DOI: https://doi.org/10.7148/2012-0490-0496
View in Google Scholar
[40] J. Restrepo, C. Gruber, and C. Machuca, “Energy profile aware routing”, in Proc. IEEE Int. Conf. on Commun. Workshops ICC 2009, Dresden, Germany, 2009, pp. 1–5, 2009. DOI: https://doi.org/10.1109/ICCW.2009.5208041
View in Google Scholar
[41] W. Fisher, M. Suchara, and J. Rexford, “Greening backbone networks: reducing energy consumption by shutting off cables in bundled links”, in Proc. 1st ACM SIGCOMM Worksh. on Green Networking Green Networking’10, New Delhi, India, 2010, pp. 29–34. DOI: https://doi.org/10.1145/1851290.1851297
View in Google Scholar
[42] F. Idzikowski, S. Orlowski, Ch. Raack, H. Rasner, and A. Wolisz, “Saving energy in IP-over-WDM networks by switching off line cards in low-demand scenarios”, in Proc. 14th Conf. on Opt. Netw. Design and Model. ONDM’10, Kyoto, Japan, 2010. DOI: https://doi.org/10.1109/ONDM.2010.5431569
View in Google Scholar
[43] N. Vasić and D. Kostić, “Energy-aware traffic engineering”, in Proc. 1st Int. Conf. on Energy-Efficient Comput. and Netw. E-ENERGY 2010, Passau, Germany, 2010. DOI: https://doi.org/10.1145/1791314.1791341
View in Google Scholar
[44] P. Arabas, K. Malinowski, and A. Sikora, “On formulation of a network energy saving optimization problem”, in Proc. of 4th Int. Conf. on Commun. and Electron. ICCE 2012, Hue, Vietnam, 2012, pp. 122–129. DOI: https://doi.org/10.1109/CCE.2012.6315903
View in Google Scholar
[45] E. Niewiadomska-Szynkiewicz et al., “Network-wide power management in computer networks”, in Proc. 22nd ITC Special. Seminar on Energy Effic. and Green Netw. SSEEGN 2013, Riccarton, New Zealand, 2013, pp. 25–30. DOI: https://doi.org/10.1109/SSEEGN.2013.6705398
View in Google Scholar
[46] E. Niewiadomska-Szynkiewicz et al., “Dynamic power management in energy-aware computer networks and data intensive systems”, Future Gener. Comp. Syst., vol. 37, pp. 284–296, 2014. DOI: https://doi.org/10.1016/j.future.2013.10.002
View in Google Scholar
[47] M. Karpowicz, P. Arabas, and E. Niewiadomska-Szynkiewicz, “Energy-aware multilevel control system for a network of Linux software routers: design and implementation”, IEEE Syst. J., vol. 12, no. 1, pp. 571–582, 2018. DOI: https://doi.org/10.1109/JSYST.2015.2489244
View in Google Scholar
[48] L. Chiaraviglio, M. Mellia, and F. Neri, “Energy-aware backbone networks: a case study”, in Proc. IEEE Int. Conf. on Commun. Worksh. ICC2009, Dresden, Germany, 2009, pp. 1–5. DOI: https://doi.org/10.1109/ICCW.2009.5208038
View in Google Scholar
[49] M. Zhang, Ch. Yi, B. Liu, and B. Zhang, “GreenTE: power-aware traffic engineering”, in Proc. IEEE Inter. Conf. on Netw. Protoc. ICNP’2010, Kyoto, Japan, 2010. DOI: https://doi.org/10.1109/ICNP.2010.5762751
View in Google Scholar
[50] G. Shen and R. S. Tucker, “Energy-minimized desig for IP over WDM networks”, J. of Optical Commun. and Netw., vol. 1, no. 1, pp. 176–186, 2009. DOI: https://doi.org/10.1364/JOCN.1.000176
View in Google Scholar
[51] R. G. Garroppo, S. Giordano, G. Nencioni, and M. G. Scutella, “Power-aware routing and network design with bundled links: Solutions and analysis”, J. of Comp. Netw. and Commun., vol. 2013, Article ID 154953, 2013. DOI: https://doi.org/10.1155/2013/154953
View in Google Scholar
[52] Y. Zhang and N. Ansari, “Hero: Hierarchical energy optimization for data center networks”, IEEE Systems J., vol. 9, no. 2, pp. 406–415, 2015. DOI: https://doi.org/10.1109/JSYST.2013.2285606
View in Google Scholar
[53] B. Heller et al., “ElasticTree: Saving energy in data center networks”, in Proc. 7th USENIX Conf. on Network. Syst. Design and Implemen. NSDI’10, San Jose, CA, USA, 2010, p. 17.
View in Google Scholar
[54] X. Wang, X. Wang, K. Zheng, Y. Yao, and Q. Cao, “Correlationaware traffic consolidation for power optimization of data center networks”, IEEE Trans. Parallel Distrib. Syst., vol. 27, no. 4, pp. 992– 1006, 2016. DOI: https://doi.org/10.1109/TPDS.2015.2421492
View in Google Scholar
[55] Z. Guo, Sh. Hui, Y. Xu, and H. J. Chao, “Dynamic flow scheduling for power-efficient data center networks”, in Proc. IEEE/ACM 24th Int. Symp. on Quality of Service IWQoS 2016, Beijing, China, 2016.
View in Google Scholar
[56] K. Zheng, X. Wang, and X. Wang, “PowerFCT: Power optimization of data center network with flow completion time constraints”, in Proc. IEEE Int. Paral. and Distrib. Proces. Symp., Hyderabad, India, 2015. DOI: https://doi.org/10.1109/IPDPS.2015.22
View in Google Scholar
[57] A. Cianfrani, V. Eramo, M. Listani, M. Marazza, and E. Vittorini, “An energy saving routing algorithm for a green OSPF protocol”, in Proc. IEEE Conf. on Comp. Commun. INFOCOM 2010, San Diego, CA, USA, 2010, pp. 1 – 5. DOI: https://doi.org/10.1109/INFCOMW.2010.5466646
View in Google Scholar
[58] A. P. Bianzino, L. Chiaraviglio, and M. Mellia, GRiDA: a green distributed algorithm for backbone networks”, in Online Conf. on Green Commun. GreenCom 2011, New York, NY, USA, 2011, pp. 113–119. DOI: https://doi.org/10.1109/GreenCom.2011.6082517
View in Google Scholar
[59] F. Cuomo, A. Abbagnale, A. Cianfrani, and M. Polverini, “Keeping the connectivity and saving the energy in the Internet”, in Proc. IEEE INFOCOM 2011 Workshop on Green Communications and Networking, Shanghai, China, 2011, pp. 319–324. DOI: https://doi.org/10.1109/INFCOMW.2011.5928831
View in Google Scholar
[60] M. Kamola and P. Arabas, “Shortest path green routing and the importance of traffic matrix knowledge”, in Proc. 24th Tyrrhenian Int. Worksh. on Digit. Commun. – Green ICT (TIWDC), Genoa, Italy, 2013. DOI: https://doi.org/10.1109/TIWDC.2013.6664215
View in Google Scholar
[61] H. Huin et al., “Bringing energy aware routing closer to reality with SDN hybrid networks”, in Proc. Global Commun. Conf. GLOBECOM’2017, Singapore, 2017, pp. 1101–1107. DOI: https://doi.org/10.1109/GLOCOM.2017.8254456
View in Google Scholar
[62] A. Verma, P. Ahuja, and A. Neogi, “pMapper: Power and migration cost aware application placement in virtualized systems”, in Middleware 2008. ACM/IFIP/USENIX 9th International Middleware Conference Leuven, Belgium, December 1-5, 2008 Proceedings, V. Issarny and R. Schantz, Eds. LNCS, vol. 5346, pp. 243–264. Springer, 2008. DOI: https://doi.org/10.1007/978-3-540-89856-6_13
View in Google Scholar
[63] A. Hameed et al., “A survey and taxonomy on energy efficient resource allocation techniques for cloud computing systems”, Computing, vol. 98, no. 7, pp. 751–774, 2016. DOI: https://doi.org/10.1007/s00607-014-0407-8
View in Google Scholar
[64] Ch. Ghribi and D. Zeghlache, “Exact and heuristic graph-coloring for energy efficient advance cloud resource reservation”, in Proc. 7th IEEE Int. Conf. on Cloud Comput. CLOUD 2014, Anchorage, AK, USA, 2014, pp. 112–119. DOI: https://doi.org/10.1109/CLOUD.2014.25
View in Google Scholar
[65] A. M. Al-Qawasmeh, S. Pasricha, A. A. Maciejewski, and H. J. Siegel, “Power and thermal-aware workload allocation in heterogeneous data centers”, IEEE Trans. on Comp., vol. 64, no. 2, pp. 477–491, 2015. DOI: https://doi.org/10.1109/TC.2013.116
View in Google Scholar
[66] K. Gupta and V. Katiyar, “Energy aware virtual machine migration techniques for cloud environment”, Int. J. of Comp. Appl., vol. 141, no. 2, pp. 11–16, 2016. DOI: https://doi.org/10.5120/ijca2016909551
View in Google Scholar
[67] V. Armant, M. De Cauwer, K. N. Brown, and B. O’Sullivan, “Semionline task assignment policies for workload consolidation in cloud computing systems”, Future Gener. Comp. Syst., vol. 82, pp. 89–103, 2018. DOI: https://doi.org/10.1016/j.future.2017.12.035
View in Google Scholar
[68] M. Zhang, S. Wang, G. Yuan, Y. Li, and Z. Qian, “Energy-efficient real-time task allocation in a data center”, in Proc. IEEE Int. Conf. on Internet of Things and Green Comput. and Commun. and Cyber, Phys. and Soc. Comput. and Smart Data iThings-GreenComCPSCom-SmartData 2016, Chengdu, China, 2016, pp. 680–687. DOI: https://doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2016.147
View in Google Scholar
[69] A. Gandhi, Yuan Ch., D. Gmach, M. Arlitt, and M. Marwah, “Minimizing data center SLA violations and power consumption via hybrid resource provisioning”, in Proc. Int. Green Comput. Conf. and Worksh. IGCC’11, Orlando, FL, USA, 2011, pp. 1–8. DOI: https://doi.org/10.1109/IGCC.2011.6008611
View in Google Scholar
[70] S. Hasan and E. Huh, “Heuristic based energy-aware resource allocation by dynamic consolidation of virtual machines in cloud data center”, TIIS, vol. 7, no. 8, pp. 1825–1842, 2013. DOI: https://doi.org/10.3837/tiis.2013.08.005
View in Google Scholar
[71] Y. Shao, C. Li, W. Dong, and Y. Liu, “Energy-aware dynamic resource allocation on hadoop YARN cluster”, in Proc. 18th Int. Conf. on High Perform. Comput. and Commun., 14th Int. Conf. on Smart City, 2nd Int. Conf. on Data Sci. and Syst. HPCC/SmartCity/DSS 2016, Sydney, NSW, Australia, 2016, pp. 364–371. DOI: https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0059
View in Google Scholar
[72] E. Niewiadomska-Szynkiewicz and P. Arabas, “Resource management system for HPC computing”, in Automation 2018. Advances in Automation, Robotics and Measurement Techniques, Warsaw, Poland, R. Szewczyk, C. Zieliński, and M. Kaliczyńska, Eds. Advances in Intelligent Systems and Computing, vol. 743, pp. 52–61. Springer, 2018. DOI: https://doi.org/10.1007/978-3-319-77179-3_5
View in Google Scholar
[73] M. Ray, S. Sondur, J. Biswas, A. Pal, and K. Kant, “Opportunistic power savings with coordinated control in data center networks”, in Proc. 19th ICDCN Int. Conf. on Distrib. Comput. and Netw., Varanasi, India, 2018, Article no. 48, pp. 48:1–48:10. DOI: https://doi.org/10.1145/3154273.3154328
View in Google Scholar
Downloads
Submitted
Published
Issue
Section
License
Copyright (c) 2018 Journal of Telecommunications and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
