C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing

Hassan A. Karimi, Duangduen Roongpiboonsopit

2011

Abstract

Interest in implementing and deploying many existing and new applications on cloud platforms is continually growing. Of these, geospatial applications, whose operations are based on geospatial data and computation, are of particular interest because they typically involve very large geospatial data layers and specialized and complex computations. In general, problems in many geospatial applications, especially those with real-time response, are compute- and/or data-intensive, which is the reason why researchers often resort to high-performance computing platforms for efficient processing. However, compared to existing high-performance computing platforms, such as grids and supercomputers, cloud computing offers new and advanced features that can benefit geospatial problem solving and application implementation and deployment. In this paper, we present a distributed algorithm for geospatial data processing on clouds and discuss the results of our experimentation with an existing cloud platform to evaluate its performance for real-time geoprocessing.

References

  1. Agarwal, S., Dunagan, J., Jain, N., Saroiu, S., Wolman, A. and Bhogan, H. (2010). Volley: Automated data placement for geo-distributed cloud services. In 7th USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA.
  2. Agarwal, S., Dunagan, J., Jain, N., Saroiu, S., Wolman, A. and Bhogan, H. (2010). Volley: Automated data placement for geo-distributed cloud services. In 7th USENIX Symposium on Networked Systems Design and Implementation (NSDI), San Jose, CA.
  3. Blower, J. (2010). GIS in the cloud: implementing a Web Map Service on Google App Engine. In 1st Intl. Conf. on Computing for Geospatial Research & Applications, Washington D.C.
  4. Blower, J. (2010). GIS in the cloud: implementing a Web Map Service on Google App Engine. In 1st Intl. Conf. on Computing for Geospatial Research & Applications, Washington D.C.
  5. Bonvin, N., Papaioannou, T. and Aberer, K. (2009). Dynamic cost-efficient replication in data clouds. In 1st workshop on Automated control for datacenters and clouds, Barcelona, Spain, 49-56.
  6. Bonvin, N., Papaioannou, T. and Aberer, K. (2009). Dynamic cost-efficient replication in data clouds. In 1st workshop on Automated control for datacenters and clouds, Barcelona, Spain, 49-56.
  7. Brauner, J., Foerster, T., Schaeffer, B. and Baranski, B. (2009). Towards a research agenda for geoprocessing services. In 12th AGILE International Conference on Geographic Information Science, Hanover, Germany.
  8. Brauner, J., Foerster, T., Schaeffer, B. and Baranski, B. (2009). Towards a research agenda for geoprocessing services. In 12th AGILE International Conference on Geographic Information Science, Hanover, Germany.
  9. Cornillon, P. (2009). Processing large volumes of satellitederived sea surface temperature data - is cloud computing the way to go? In Cloud Computing and Collaborative Technologies in the Geosciences Workshop, Indianapolis, IN.
  10. Cornillon, P. (2009). Processing large volumes of satellitederived sea surface temperature data - is cloud computing the way to go? In Cloud Computing and Collaborative Technologies in the Geosciences Workshop, Indianapolis, IN.
  11. ESRI, (2009). Spatial data service deployment utility for Windows Azure is available! Retrieved from: http:// blogs.esri.com/Dev/blogs/mapit/archive/2009/12/18/S patial-Data-Service-Deployment-Utility-for-WindowsAzure-is-available_2100.aspx.
  12. ESRI, (2009). Spatial data service deployment utility for Windows Azure is available! Retrieved from: http:// blogs.esri.com/Dev/blogs/mapit/archive/2009/12/18/S patial-Data-Service-Deployment-Utility-for-WindowsAzure-is-available_2100.aspx.
  13. ESRI, (2010). ArcGIS and the cloud Retrieved from: http://www.esri.com/technology-topics/cloud-gis/arc gis-and-the-cloud.html.
  14. ESRI, (2010). ArcGIS and the cloud Retrieved from: http://www.esri.com/technology-topics/cloud-gis/arc gis-and-the-cloud.html.
  15. Finkel, R., Bentley, J. (1974). Quad trees a data structure for retrieval on composite keys. Acta informatica, 4(1), 1-9.
  16. Finkel, R., Bentley, J. (1974). Quad trees a data structure for retrieval on composite keys. Acta informatica, 4(1), 1-9.
  17. Foerster, T., Schaeffer, B., Baranski, B. and Lange, K. (2010). Geoprocessing in hybrid clouds. In Geoinformatik, Kiel, Germany.
  18. Foerster, T., Schaeffer, B., Baranski, B. and Lange, K. (2010). Geoprocessing in hybrid clouds. In Geoinformatik, Kiel, Germany.
  19. Guttman, A. (1984). R-trees: a dynamic index structure for spatial searching. In Proceedings of the 1984 ACM SIGMOD international conference on Management of data, Boston, Massachusetts, 47-57.
  20. Guttman, A. (1984). R-trees: a dynamic index structure for spatial searching. In Proceedings of the 1984 ACM SIGMOD international conference on Management of data, Boston, Massachusetts, 47-57.
  21. Hill, C. (2009). Experiences with atmosphere and ocean models on EC2. In Cloud Computing and Collaborative Technologies in the Geosciences Workshop, Indianapolis, IN.
  22. Hill, C. (2009). Experiences with atmosphere and ocean models on EC2. In Cloud Computing and Collaborative Technologies in the Geosciences Workshop, Indianapolis, IN.
  23. Hunter, G. (1978). Efficient computation and data structures for graphics. Princeton University, Princeton, NJ, USA.
  24. Hunter, G. (1978). Efficient computation and data structures for graphics. Princeton University, Princeton, NJ, USA.
  25. Karimi, H. A., Hwang, D. (1997). A Parallel Algorithm for Routing: Best Solutions at Low Computational Costs. Geomatica, 51(1), 45-51.
  26. Karimi, H. A., Hwang, D. (1997). A Parallel Algorithm for Routing: Best Solutions at Low Computational Costs. Geomatica, 51(1), 45-51.
  27. Kim, K. S., MacKenzie, D. (2009). Use of cloud computing in impact assessment of climate change. In Free and Open Source Software for Geospatial (FOSS4GT), Sydney, Australia.
  28. Kim, K. S., MacKenzie, D. (2009). Use of cloud computing in impact assessment of climate change. In Free and Open Source Software for Geospatial (FOSS4GT), Sydney, Australia.
  29. Liu, S., Karimi, H. (2008). Grid query optimizer to improve query processing in grids. Future Generation Computer Systems, 24(5), 342-353.
  30. Liu, S., Karimi, H. (2008). Grid query optimizer to improve query processing in grids. Future Generation Computer Systems, 24(5), 342-353.
  31. Mackert, L. F., Lohman, G. M. (1986). R* Optimizer Validation and Performance Evaluation for Distributed Queries. In the Twelfth International Conference on Very Large Data Bases, Kyoto.
  32. Mackert, L. F., Lohman, G. M. (1986). R* Optimizer Validation and Performance Evaluation for Distributed Queries. In the Twelfth International Conference on Very Large Data Bases, Kyoto.
  33. Mouza, C. D., Litwin, W. and Rigaux, P. (2007). SDRtree: A scalable distributed Rtree. In IEEE 23rd International Conference on Data Engineering (ICDE), Istanbul, Turkey, 296-305.
  34. Mouza, C. D., Litwin, W. and Rigaux, P. (2007). SDRtree: A scalable distributed Rtree. In IEEE 23rd International Conference on Data Engineering (ICDE), Istanbul, Turkey, 296-305.
  35. Mouza, C. D., Litwin, W. and Rigaux, P. (2009). Largescale indexing of spatial data in distributed repositories: the SD-Rtree. The VLDB Journal, 18(4), 933-958.
  36. Mouza, C. D., Litwin, W. and Rigaux, P. (2009). Largescale indexing of spatial data in distributed repositories: the SD-Rtree. The VLDB Journal, 18(4), 933-958.
  37. Nurik, R., Shen, S., (2009). Geospatial Queries with Google App Engine using GeoModel Retrieved from: http://code.google.com/apis/maps/articles/geo spatial.html#geomodel.
  38. Nurik, R., Shen, S., (2009). Geospatial Queries with Google App Engine using GeoModel Retrieved from: http://code.google.com/apis/maps/articles/geo spatial.html#geomodel.
  39. Omnisdata, (2010). GIS Cloud beta: the next generation of GIS Retrieved from: http://www.giscloud.com/.
  40. Omnisdata, (2010). GIS Cloud beta: the next generation of GIS Retrieved from: http://www.giscloud.com/.
  41. Ratnasamy, S., Francis, P., Handley, M., Karp, R. and Schenker, S. (2001). A scalable content-addressable network. In ACM SIGCOMM Computer Communication Review, San Diego, CA, USA, 161- 172.
  42. Ratnasamy, S., Francis, P., Handley, M., Karp, R. and Schenker, S. (2001). A scalable content-addressable network. In ACM SIGCOMM Computer Communication Review, San Diego, CA, USA, 161- 172.
  43. Reddy, D., Rubin, S., 1978. Representation of threedimensional objects (No. CMU-CS-78-113). Pittsburgh, PA: Computer Science Department, Carnegie-Mellon University.
  44. Reddy, D., Rubin, S., 1978. Representation of threedimensional objects (No. CMU-CS-78-113). Pittsburgh, PA: Computer Science Department, Carnegie-Mellon University.
  45. Robinson, J. (1981). The KDB-tree: a search structure for large multidimensional dynamic indexes. In Proceedings of the 1981 ACM SIGMOD International Conference on Management of Data, Ann Arbor, Michigan, 10-18.
  46. Robinson, J. (1981). The KDB-tree: a search structure for large multidimensional dynamic indexes. In Proceedings of the 1981 ACM SIGMOD International Conference on Management of Data, Ann Arbor, Michigan, 10-18.
  47. Samet, H. (1984). The quadtree and related hierarchical data structures. ACM Computing Surveys (CSUR), 16(2), 187-260.
  48. Samet, H. (1984). The quadtree and related hierarchical data structures. ACM Computing Surveys (CSUR), 16(2), 187-260.
  49. Sato, K., Sato, H. and Matsuoka, S. (2009). A modelbased algorithm for optimizing I/O intensive applications in clouds using VM-based migration. In 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID), Shanghai, China, 466-471.
  50. Sato, K., Sato, H. and Matsuoka, S. (2009). A modelbased algorithm for optimizing I/O intensive applications in clouds using VM-based migration. In 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID), Shanghai, China, 466-471.
  51. Schäffer, B., Baranski, B. (2009). Towards spatial related business processes in SDIs. In 12th AGILE International Conference on Geographic Information Science, Hannover, Germany.
  52. Schäffer, B., Baranski, B. (2009). Towards spatial related business processes in SDIs. In 12th AGILE International Conference on Geographic Information Science, Hannover, Germany.
  53. Voicu, L. C., Schuldt, H., Breitbart, Y. and Schek, H.-J. (2010). Data and flexible data access in a cloud based on freshness requirements. In 3rd IEEE International Conference on Cloud Computing (CLOUD2010), Miami, FL, USA, 45-48.
  54. Voicu, L. C., Schuldt, H., Breitbart, Y. and Schek, H.-J. (2010). Data and flexible data access in a cloud based on freshness requirements. In 3rd IEEE International Conference on Cloud Computing (CLOUD2010), Miami, FL, USA, 45-48.
  55. Wang, J., Wu, S., Gao, H., Li, J. and Ooi, B. C. (2010). Indexing multi-dimensional data in a cloud system. In ACM SIGMOD/PODS Conference, Indianapolis, IN, USA.
  56. Wang, J., Wu, S., Gao, H., Li, J. and Ooi, B. C. (2010). Indexing multi-dimensional data in a cloud system. In ACM SIGMOD/PODS Conference, Indianapolis, IN, USA.
  57. Wang, Y., Wang, S. and Zhou, D. (2009). Retrieving and indexing spatial data in the cloud computing environment. Lecture Notes in Computer Science, Cloud Computing, 322-331.
  58. Wang, Y., Wang, S. and Zhou, D. (2009). Retrieving and indexing spatial data in the cloud computing environment. Lecture Notes in Computer Science, Cloud Computing, 322-331.
  59. Williams, H. (2009). A new paradigm for geographic information services. Spatial Cloud Computing (SC2), White Paper.
  60. Williams, H. (2009). A new paradigm for geographic information services. Spatial Cloud Computing (SC2), White Paper.
  61. Wu, S., Wu, K.-L. (2009). An indexing framework for efficient retrieval on the cloud. IEEE Data Engineering, 32(1), 75-82.
  62. Wu, S., Wu, K.-L. (2009). An indexing framework for efficient retrieval on the cloud. IEEE Data Engineering, 32(1), 75-82.
  63. Zimmermann, R., Ku, W. and Chu, W. (2004). Efficient query routing in distributed spatial databases. In 12th annual ACM international workshop on Geographic information systems, Washington DC, USA, 176-183.
  64. Zimmermann, R., Ku, W. and Chu, W. (2004). Efficient query routing in distributed spatial databases. In 12th annual ACM international workshop on Geographic information systems, Washington DC, USA, 176-183.
Download


Paper Citation


in Harvard Style

A. Karimi H. and Roongpiboonsopit D. (2011). C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing . In Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-8425-52-2, pages 371-381. DOI: 10.5220/0003394203710381


in Harvard Style

A. Karimi H. and Roongpiboonsopit D. (2011). C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing . In Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-8425-52-2, pages 371-381. DOI: 10.5220/0003394203710381


in Bibtex Style

@conference{closer11,
author={Hassan A. Karimi and Duangduen Roongpiboonsopit},
title={C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing},
booktitle={Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2011},
pages={371-381},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003394203710381},
isbn={978-989-8425-52-2},
}


in Bibtex Style

@conference{closer11,
author={Hassan A. Karimi and Duangduen Roongpiboonsopit},
title={C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing},
booktitle={Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2011},
pages={371-381},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003394203710381},
isbn={978-989-8425-52-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing
SN - 978-989-8425-52-2
AU - A. Karimi H.
AU - Roongpiboonsopit D.
PY - 2011
SP - 371
EP - 381
DO - 10.5220/0003394203710381


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - C2GEO - Techniques and Tools for Real-time Data-intensive Geoprocessing in Cloud Computing
SN - 978-989-8425-52-2
AU - A. Karimi H.
AU - Roongpiboonsopit D.
PY - 2011
SP - 371
EP - 381
DO - 10.5220/0003394203710381