Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 11

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems

The Next Generation of In-home Streaming: Light Fields, 5K, 10 GbE, and Foveated Compression

, , , , , ,

DOI: http://dx.doi.org/10.15439/2017F16

Citation: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 11, pages 663–667 ()

Full text

Abstract. Interacting with real-time rendered 3D content from powerful machines on smaller devices is becoming ubiquitous through commercial products that enable in-home streaming within the same local network. However, support for high resolution, low latency in-home streaming at high image quality is still a challenging problem. To enable this, we enhance an existing open source framework for in-home streaming. We add highly optimized DXT1 (DirectX Texture Compression) support for thin desktop and notebook clients. For rendered light fields, we improve the encoding algorithms for higher image quality. Within a 10 Gigabit Ethernet (10 GbE) network, we achieve streaming up to 5K resolution at 55 frames per second. Through new low-level algorithmic improvements, we increase the compression speed of ETC1 (Ericsson Texture Compression) by a factor of 5. We are the first to bring ETC2 compression to real-time speed, which increases the streamed image quality. Last, we reduce the required data rate by more than a factor of 2 through foveated compression with real-time eye tracking.

References

  1. M. Claypool and K. Claypool, “Latency and player actions in online games”, Comm. of the ACM, vol. 49, no. 11, pp. 40–45, 2006.
  2. D. Pohl, B. Taudul, R. Membarth, S. Nickels, and O. Grau, “Advanced in-home streaming to mobile devices and wearables”, IJCSA, vol. 12, no. 2, 2015.
  3. T. Richardson, Q. Stafford-Fraser, K. Wood, and A. Hopper, “Virtual network computing”, IEEE Internet Computing, vol. 2, no. 1, pp. 33–38, 1998. http://dx.doi.org/10.1109/4236.656066.
  4. T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard”, Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560–576, 2003.
  5. A. Travis, “Autostereoscopic displays”, in Handbook of Visual Display Technology. Springer, 2012, pp. 1861–1873.
  6. Wireless LAN Working Group, IEEE Standard 802.11ac-2013 (Amendment to IEEE Std 802.11-2012), Dec. 2013.
  7. J. Ström and T. Akenine-Möller, “Ipackman: High-quality, low-complexity texture compression for mobile phones”, 2005, pp. 63–70. DOI : 10.1145/1071866.1071877.
  8. J. Ström and M. Pettersson, “ETC2: Texture compression using invalid combinations”, in Graphics Hardware, 2007, pp. 49–54.
  9. P. Brown, I. Stewart, N. Haemel, A. Pooley, A. Rasmus, and M. Shah, OpenGL S3TC extension spec, 2000.
  10. B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder, “Foveated 3D graphics”, ACM Transactions on Graphics, vol. 31, no. 6, p. 164, 2012.
  11. F. Zund, Y. Pritch, A. Sorkine-Hornung, S. Mangold, and T. Gross, “Content-aware compression using saliency-driven image retargeting”, in ICIP, 2013, pp. 1845–1849.
  12. C. Koch and S. Ullman, “Shifts in selective visual attention: Towards the underlying neural circuitry”, in Matters of Intelligence, Springer, 1987, pp. 115–141.
  13. L. Renambot, B. Jeong, and J. Leigh, “Real-time compression for high-resolution content”, Proceedings of the Access Grid Retreat, vol. 7, 2007.
  14. I. Wald, S. Woop, C. Benthin, G. S. Johnson, and M. Ernst, “Embree: A kernel framework for efficient cpu ray tracing”, ACM Transactions on Graphics, vol. 33, no. 4, 143:1–143:8, 2014.
  15. Y. Wang, J. Ostermann, and Y. Zhang, Video Processing and Communications. Prentice Hall, 2002.
  16. Z. Wang, L. Lu, and A. Bovik, “Video quality assessment based on structural distortion measurement”, Signal Processing: Image Communication, vol. 19, no. 2, pp. 121–132, 2004. http://dx.doi.org/10.1016/S0923-5965(03)00076-6.
  17. G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard”, Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649–1668, 2012.
  18. C. Taylor, “802.11ac Wave 2 with MU-MIMO: The next mainstream Wi-Fi standard”, 2015.