Academic Positions

  • now 2015

    Postdoctoral Fellow

    Land Surveying and Geo-Informatics
    The Hong Kong Polytechnique University

  • 2013 2011

    Research Assistant

    Land Surveying and Geo-Informatics
    The Hong Kong Polytechnique University

Education & Training

  • 2015 2010

    PhD. student

    State Key Laboratory of Surveying Mapping and Remote Sensing
    Wuhan University

  • 2010 2006

    Undergraduate student

    School of Remote Sensing and Information Enginnering
    Wuhan University

Honors, Awards and Grants

  • 2018
    The Emerging Star Award of GIS by the 7th College GIS Forum (高校GIS新锐奖)

    This award is granted to the top 10 youth college staffs in the GIS field.

  • 2016
    Special Merit Award (R. Alekseev Award) and Gold Medal of the 44th International Exhibition of Inventions of Geneva

    The International Exhibition of Inventions of Geneva is one of the most important specialized and large-scale invention exhibitions in the world. It is supported by the Swiss Federal Government, the State and the City of Geneva, as well as the World Intellectual Property Organization. The exhibition this year attracted more than 700 producers and inventors from 45 countries and over 60000 visitors during the five days exhibition from April 13 to 17, 2016.

    The award is presented to our project, "Precise Topographic Mapping Model", in which we developped a integrated combined adjustment model for the lunar topographic mapping. This method has been used in China's previous lunar exploration missions. Please see our papers:

    Wu, B., Hu, H.*, Guo, J., 2014. Integration of Chang'E-2 imagery and LRO laser altimeter data with a combined block adjustment for precision lunar topographic modeling. Earth and Planetary Science Letters 391, 1-15.

  • 2016
    the Second Honorable Mention for the 2016 Talbert Abrams Award

    The purpose of the Award is to encourage the authorship and recording of current, historical, engineering, and scientific developments in photogrammetry. Three papers from 12 issues of the 2015 PE&RS journal are selected. The paper being awarded is:

    Hu, H.*, Zhu, Q., Du, Z., Zhang, Y., Ding, Y., 2015. Reliable spatial relationship constrained feature point matching of oblique aerial images. Photogrammetric Engineering and Remote Sensing 81 (1), 49-58.

  • Sept. 2014
    National Scholarship
    Rank 2 out of more than 50 PhD. students in the State Key Laboratory of Surveying Mapping and Remote Sensing for the National Scholarship award.
  • Mar. 2014
    John I. Davidson President’s Award
    This Award is presented by ASPRS with funding provided by the ASPRS Foundation to to encourage and commend those who publish papers of practical or applied value in PE&RS, the official journal of ASPRS.

    This award is granted to our paper: Wu, B., Hu, H., Zhu, Q., Zhang, Y., 2013. A flexible method for zoom lens calibration and modeling using a planar checkerboard. Photogrammetric engineering and remote sensing 79 (6), 555-571.
  • Sept. 2010
    The Freshman Scholarship
    Rank 1st in the graduate entrance examination for the State Key Laboratory of Surveying Mapping and Remote Sensing. And this shcolarship is granted to six out of more than 100 graduate students.
  • Sept. 2008
    Kwang-Hua Scholarship
    Top 30% of the undergraduate students in School of Remote Sensing and Information Enginnering.

Research Projects

  • image

    Research project

    • 2021/12 2017/01

      National Natural Science Foundation of China (自然科学基金青年项目)

      A supervised metric learning approach for efficient aerial oblique image matching using neighborhood information, PI

    • 2021/12 2017/01

      National Natural Science Foundation of China (自然科学基金重点项目)

      Theory and methods of the oblique photogrammetry for precision building reconstruction, Co-I

    • 2018/04 2016/11

      Open Fund of Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Land and Resources

      LOD reconstruction of buildings from oblique images, PI

Filter by year:

Sort by year:

Hierarchical Regularization of Building Boundaries in Noisy Aerial Laser Scanning and Photogrammetric Point Clouds

Xie, L., Zhu, Q., Hu, H.*, Wu, B., Li, Y., Zhang, Y., Zhong, R., 2018.
Journal Papers Remote Sensing 10 (12), 1996.

Abstract

Aerial laser scanning or photogrammetric point clouds are often noisy at building boundaries. In order to produce regularized polygons from such noisy point clouds, this study proposes a hierarchical regularization method for the boundary points. Beginning with detected planar structures from raw point clouds, two stages of regularization are employed. In the first stage, the boundary points of an individual plane are consolidated locally by shifting them along their refined normal vector to resist noise, and then grouped into piecewise smooth segments. In the second stage, global regularities among different segments from different planes are softly enforced through a labeling process, in which the same label represents parallel or orthogonal segments. This is formulated as a Markov random field and solved efficiently via graph cut. The performance of the proposed method is evaluated for extracting 2D footprints and 3D polygons of buildings in metropolitan area. The results reveal that the proposed method is superior to the state-of-art methods both qualitatively and quantitatively in compactness. The simplified polygons could fit the original boundary points with an average residuals of 0.2 m, and in the meantime reduce up to 90% complexities of the edges. The satisfactory performances of the proposed method show a promising potential for 3D reconstruction of polygonal models from noisy point clouds.

Intact Planar Abstraction of Buildings via Global Normal Refinement from Noisy Oblique Photogrammetric Point Clouds

Zhu, Q., Wang, F., Hu, H.*, Ding, Y., Xie, J., Wang, W., Zhong, R., 2018
Journal Papers ISPRS International Journal of Geo-Information 7 (11), 431-452

Abstract

The second column is the normal vector estimated through the proposed method, see the sharp feature preserved on the building edge.

Oblique photogrammetric point clouds are currently one of the major data sources for the three-dimensional level-of-detail reconstruction of buildings. However, they are severely noise-laden and pose serious problems for the effective and automatic surface extraction of buildings. In addition, conventional methods generally use normal vectors estimated in a local neighborhood, which are liable to be affected by noise, leading to inferior results in successive building reconstruction. In this paper, we propose an intact planar abstraction method for buildings, which explicitly handles noise by integrating information in a larger context through global optimization. The information propagates hierarchically from a local to global scale through the following steps: first, based on voxel cloud connectivity segmentation, single points are clustered into supervoxels that are enforced to not cross the surface boundary; second, each supervoxel is expanded to nearby supervoxels through the maximal support region, which strictly enforces planarity; third, the relationships established by the maximal support regions are injected into a global optimization, which reorients the local normal vectors to be more consistent in a larger context; finally, the intact planar surfaces are obtained by region growing using robust normal and point connectivity in the established spatial relations. Experiments on the photogrammetric point clouds obtained from oblique images showed that the proposed method is effective in reducing the influence of noise and retrieving almost all of the major planar structures of the examined buildings.

Block adjustment and coupled epipolar rectification of LROC NAC images for precision lunar topographic mapping

Hu, H. and B. Wu., 2018
Journal Papers Planetary and Space Science 160, 26-38

Abstract

Anaglyph on the border area between NAC-L and NAC-R
Example of NAC DEMs
Example of NAC DEMs

The narrow-angle camera (NAC) of the lunar reconnaissance orbiter camera (LROC) uses a pair of closely attached pushbroom sensors to obtain a large swath of coverage while providing high-resolution imaging. However, the two image sensors do not share the same lenses and cannot be modelled geometrically with a single physical model. The irregular image network leads to difficulties in conducting the block adjustment to remove inconsistencies between the NAC images in both intra- and inter-track cases. In addition, the special image network requires two to four stereo models, each with an irregular overlapping region of varying size. The stereo configuration of NAC images also creates severe problems for the state-of-the-art image matching methods. With the aim of using NAC stereo pairs for precision 3D topographic mapping, this paper presents a novel approach to the block adjustment and coupled epipolar rectification of NAC stereo images. This approach removes the internal inconsistencies in a single block adjustment and merges the image pair in the disparity space, thus requiring estimation of only one stereo model. Semi-global matching is used to generate a disparity map for the stereo pair; each correspondence is transformed back to the source image, and 3D points are derived via photogrammetric space intersection. The experimental results reveal that the proposed approach is able to reduce the gaps and inconsistencies caused by the inaccurate boresight offsets between the two NAC cameras and the irregular overlapping regions and to finally generate precise and consistent 3D topographic models from the NAC stereo images.

Illumination invariant feature point matching for high-resolution planetary remote sensing images

Wu, B., Zeng, H., Hu, H.
Journal Papers Planetary and Space Science 152, 45-54.

Abstract

Despite its success with regular close-range and remote-sensing images, the scale-invariant feature transform (SIFT) algorithm is essentially not invariant to illumination differences due to the use of gradients for feature description. In planetary remote sensing imagery, which normally lacks sufficient textural information, salient regions are generally triggered by the shadow effects of keypoints, reducing the matching performance of classical SIFT. Based on the observation of dual peaks in a histogram of the dominant orientations of SIFT keypoints, this paper proposes an illumination-invariant SIFT matching method for high-resolution planetary remote sensing images. First, as the peaks in the orientation histogram are generally aligned closely with the sub-solar azimuth angle at the time of image collection, an adaptive suppression Gaussian function is tuned to level the histogram and thereby alleviate the differences in illumination caused by a changing solar angle. Next, the suppression function is incorporated into the original SIFT procedure for obtaining feature descriptors, which are used for initial image matching. Finally, as the distribution of feature descriptors changes after anisotropic suppression, and the ratio check used for matching and outlier removal in classical SIFT may produce inferior results, this paper proposes an improved matching procedure based on cross-checking and template image matching. The experimental results for several high-resolution remote sensing images from both the Moon and Mars, with illumination differences of 20°–180°, reveal that the proposed method retrieves about 40%–60% more matches than the classical SIFT method. The proposed method is of significance for matching or co-registration of planetary remote sensing images for their synergistic use in various applications. It also has the potential to be useful for flyby and rover images by integrating with the affine invariant feature detectors.

Integration of aerial oblique imagery and terrestrial imagery for optimized 3D modeling in urban areas

Wu, B., Xie, L., Hu, H., Zhu, Q., Yau, E., 2018
Journal Papers ISPRS Journal of Photogrammetry and Remote Sensing 139, 119-132.

Abstract

Photorealistic three-dimensional (3D) models are fundamental to the spatial data infrastructure of a digital city, and have numerous potential applications in areas such as urban planning, urban management, urban monitoring, and urban environmental studies. Recent developments in aerial oblique photogrammetry based on aircraft or unmanned aerial vehicles (UAVs) offer promising techniques for 3D modeling. However, 3D models generated from aerial oblique imagery in urban areas with densely distributed high-rise buildings may show geometric defects and blurred textures, especially on building façades, due to problems such as occlusion and large camera tilt angles. Meanwhile, mobile mapping systems (MMSs) can capture terrestrial images of close-range objects from a complementary view on the ground at a high level of detail, but do not offer full coverage. The integration of aerial oblique imagery with terrestrial imagery offers promising opportunities to optimize 3D modeling in urban areas. This paper presents a novel method of integrating these two image types through automatic feature matching and combined bundle adjustment between them, and based on the integrated results to optimize the geometry and texture of the 3D models generated from aerial oblique imagery. Experimental analyses were conducted on two datasets of aerial and terrestrial images collected in Dortmund, Germany and in Hong Kong. The results indicate that the proposed approach effectively integrates images from the two platforms and thereby improves 3D modeling in urban areas.

Geometric Integration of Hybrid Correspondences for RGB-D Unidirectional Tracking

Tang, S., Chen, W., Wang, W., Li, X., Darwish, W., Li, W., Huang, Z., Hu, H., Guo, R., 2018
Journal Papers Sensors 18 (5): 1385.

Abstract

Traditionally, visual-based RGB-D SLAM systems only use correspondences with valid depth values for camera tracking, thus ignoring the regions without 3D information. Due to the strict limitation on measurement distance and view angle, such systems adopt only short-range constraints which may introduce larger drift errors during long-distance unidirectional tracking. In this paper, we propose a novel geometric integration method that makes use of both 2D and 3D correspondences for RGB-D tracking. Our method handles the problem by exploring visual features both when depth information is available and when it is unknown. The system comprises two parts: coarse pose tracking with 3D correspondences, and geometric integration with hybrid correspondences. First, the coarse pose tracking generates the initial camera pose using 3D correspondences with frame-by-frame registration. The initial camera poses are then used as inputs for the geometric integration model, along with 3D correspondences, 2D-3D correspondences and 2D correspondences identified from frame pairs. The initial 3D location of the correspondence is determined in two ways, from depth image and by using the initial poses to triangulate. The model improves the camera poses and decreases drift error during long-distance RGB-D tracking iteratively. Experiments were conducted using data sequences collected by commercial Structure Sensors. The results verify that the geometric integration of hybrid correspondences effectively decreases the drift error and improves mapping accuracy. Furthermore, the model enables a comparative and synergistic use of datasets, including both 2D and 3D features

Road Extraction from VHR Remote-Sensing Imagery via Object Segmentation Constrained by Gabor Features

Chen, L., Zhu, Q., Xie, X., Hu, H., Zeng, H., 2018.
Journal Papers ISPRS International Journal of Geo-Information 7 (9), 362.

Abstract

Automatic road extraction from remote-sensing imagery plays an important role in many applications. However, accurate and efficient extraction from very high-resolution (VHR) images remains difficult because of, for example, increased data size and superfluous details, the spatial and spectral diversity of road targets, disturbances (e.g., vehicles, shadows of trees, and buildings), the necessity of finding weak road edges while avoiding noise, and the fast-acquisition requirement of road information for crisis response. To solve these difficulties, a two-stage method combining edge information and region characteristics is presented. In the first stage, convolutions are executed by applying Gabor wavelets in the best scale to detect Gabor features with location and orientation information. The features are then merged into one response map for connection analysis. In the second stage, highly complete, connected Gabor features are used as edge constraints to facilitate stable object segmentation and limit region growing. Finally, segmented objects are evaluated by some fundamental shape features to eliminate nonroad objects. The results indicate the validity and superiority of the proposed method to efficiently extract accurate road targets from VHR remote-sensing images

Precision 3D surface reconstruction from LRO NAC images using semi-global matching with coupled epipolar rectification

Hu, H. and B. Wu.*, 2017
Conference Papers ISPRS Archives, the International Symposium on Planetary Remote Sensing and Mapping, 13–16 August 2017, Hong Kong

Abstract

The Narrow-Angle Camera (NAC) on board the Lunar Reconnaissance Orbiter (LRO) comprises of a pair of closely attached high-resolution push-broom sensors, in order to improve the swath coverage. However, the two image sensors do not share the same lenses and cannot be modelled geometrically using a single physical model. Thus, previous works on dense matching of stereo pairs of NAC images would generally create two to four stereo models, each with an irregular and overlapping region of varying size. Semi-Global Matching (SGM) is a well-known dense matching method and has been widely used for image-based 3D surface reconstruction. SGM is a global matching algorithm relying on global inference in a larger context rather than individual pixels to establish stable correspondences. The stereo configuration of LRO NAC images causes severe problem for image matching methods such as SGM, which emphasizes global matching strategy. Aiming at using SGM for image matching of LRO NAC stereo pairs for precision 3D surface reconstruction, this paper presents a coupled epipolar rectification methods for LRO NAC stereo images, which merges the image pair in the disparity space and in this way, only one stereo model will be estimated. For a stereo pair (four) of NAC images, the method starts with the boresight calibration by finding correspondence in the small overlapping stripe between each pair of NAC images and bundle adjustment of the stereo pair, in order to clean the vertical disparities. Then, the dominate direction of the images are estimated by project the center of the coverage area to the reference image and back-projected to the bounding box plane determined by the image orientation parameters iteratively. The dominate direction will determine an affine model, by which the pair of NAC images are warped onto the object space with a given ground resolution and in the meantime, a mask is produced indicating the owner of each pixel. SGM is then used to generate a disparity map for the stereo pair and each correspondence is transformed back to the owner and 3D points are derived through photogrammetric space intersection. Experimental results reveal that the proposed method is able to reduce gaps and inconsistencies caused by the inaccurate boresight offsets between the two NAC cameras and the irregular overlapping regions, and finally generate precise and consistent 3D surface models from the NAC stereo images automatically.

Hierarchical Regularization of Polygons for Photogrammetric Point Clouds of Oblique Images

Xie, L., Hu, H., Zhu, Q., Wu, B., Zhang, Y., 2017
Conference Papers ISPRS Hannover Workshop: HRIGI 17 – CMRT 17 – ISA 17 – EuroCOW 17, 6–9 June 2017, Hannover, Germany

Abstract

Comparison of regularization results: proposed (top), Douglas-Peuker (middle), Dyken (bottom)

Despite the success of multi-view stereo (MVS) reconstruction from massive oblique images in city scale, only point clouds and triangulated meshes are available from existing MVS pipelines, which are topologically defect laden, free of semantical information and hard to edit and manipulate interactively in further applications. On the other hand, 2D polygons and polygonal models are still the industrial standard. However, extraction of the 2D polygons from MVS point clouds is still a non-trivial task, given the fact that the boundaries of the detected planes are zigzagged and regularities, such as parallel and orthogonal, cannot preserve. Aiming to solve these issues, this paper proposes a hierarchical polygon regularization method for the photogrammetric point clouds from existing MVS pipelines, which comprises of local and global levels. After boundary points extraction, e.g. using alpha shapes, the local level is used to consolidate the original points, by refining the orientation and position of the points using linear priors. The points are then grouped into local segments by forward searching. In the global level, regularities are enforced through a labeling process, which encourage the segments share the same label and the same label represents segments are parallel or orthogonal. This is formulated as Markov Random Field and solved efficiently. Preliminary results are made with point clouds from aerial oblique images and compared with two classical regularization methods, which have revealed that the proposed method are more powerful in abstracting a single building and is promising for further 3D polygonal model reconstruction and GIS applications.

Robust point cloud classification based on multi-level semantic relationships for urban scenes

Zhu, Q., Li, Y., Hu, H.*, Wu, B., 2017
Journal Papers ISPRS Journal of Photogrammetry and Remote Sensing 129, 86-102

Abstract

Classification results of 15 dense-matched point cloud tiles

The semantic classification of point clouds is a fundamental part of three-dimensional urban reconstruction. For datasets with high spatial resolution but significantly more noises, a general trend is to exploit more contexture information to surmount the decrease of discrimination of features for classification. However, previous works on adoption of contexture information are either too restrictive or only in a small region and in this paper, we propose a point cloud classification method based on multi-level semantic relationships, including point–homogeneity, supervoxel–adjacency and class–knowledge constraints, which is more versatile and incrementally propagate the classification cues from individual points to the object level and formulate them as a graphical model. The point–homogeneity constraint clusters points with similar geometric and radiometric properties into regular-shaped supervoxels that correspond to the vertices in the graphical model. The supervoxel–adjacency constraint contributes to the pairwise interactions by providing explicit adjacent relationships between supervoxels. The class–knowledge constraint operates at the object level based on semantic rules, guaranteeing the classification correctness of supervoxel clusters at that level. International Society of Photogrammetry and Remote Sensing (ISPRS) benchmark tests have shown that the proposed method achieves state-of-the-art performance with an average per-area completeness and correctness of 93.88% and 95.78%, respectively. The evaluation of classification of photogrammetric point clouds and DSM generated from aerial imagery confirms the method’s reliability in several challenging urban scenes.

An Adaptive Dense Matching Methodfor AirborneImages Using Texture Information

Zhu, Q., Chen, C., Hu, H.*, Ding, Y., 2017
Journal Papers Acta Geodaetica et Cartographica Sinica 46(1), 62-72

Abstract

Semi-global matching (SGM) is essentially a discrete optimization for the disparity value of each pixel, under the assumption of disparity continuities. SGM overcomes the influence of the disparity discontinuities by a set of parameters. Using smaller parameters, the continuity constraint is weakened, which will cause significant noises in planar and textureless areas, reflected as the fluctuations on the final surface reconstruction. On the other hands, larger parameters will impose too much constraints on continuities, which may lead to losses of sharp features. To address this problem, this paper proposes an adaptive dense stereo matching methods for airborne images using with texture information. Firstly, the texture is quantified, and under the assumption that disparity variation is directly proportional to the texture information, the adaptive parameters are gauged accordingly. Second, SGM is adopted to optimize the discrete disparities using the adaptively tuned parameters. Experimental evaluations using the ISPRS benchmark dataset and images obtained by the SWDC-5 have revealed that the proposed method will significantly improve the visual qualities of the point clouds.

Enhanced RGB-D Mapping Method for Detailed 3D Indoor and Outdoor Modeling

Tang, S., Zhu, Q., Chen, W., Darwish, W., Wu, B., Hu, H., Chen, M., 2016
Journal PapersSensors 16 (10), 1589.

Abstract

RGB-D sensors (sensors with RGB camera and Depth camera) are novel sensing systems that capture RGB images along with pixel-wise depth information. Although they are widely used in various applications, RGB-D sensors have significant drawbacks including limited measurement ranges (e.g., within 3 m) and errors in depth measurement increase with distance from the sensor with respect to 3D dense mapping. In this paper, we present a novel approach to geometrically integrate the depth scene and RGB scene to enlarge the measurement distance of RGB-D sensors and enrich the details of model generated from depth images. First, precise calibration for RGB-D Sensors is introduced. In addition to the calibration of internal and external parameters for both, IR camera and RGB camera, the relative pose between RGB camera and IR camera is also calibrated. Second, to ensure poses accuracy of RGB images, a refined false features matches rejection method is introduced by combining the depth information and initial camera poses between frames of the RGB-D sensor. Then, a global optimization model is used to improve the accuracy of the camera pose, decreasing the inconsistencies between the depth frames in advance. In order to eliminate the geometric inconsistencies between RGB scene and depth scene, the scale ambiguity problem encountered during the pose estimation with RGB image sequences can be resolved by integrating the depth and visual information and a robust rigid-transformation recovery method is developed to register RGB scene to depth scene. The benefit of the proposed joint optimization method is firstly evaluated with the publicly available benchmark datasets collected with Kinect. Then, the proposed method is examined by tests with two sets of datasets collected in both outside and inside environments. The experimental results demonstrate the feasibility and robustness of the proposed method.

Texture-Aware Dense Image Matching using Ternary Census Transform

Hu, H.*, Chen, C., Wu, B., Yang, X., Zhu, Q., Ding, Y., 2016
Conference PapersISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences III-3, 59-66, Prague Czech, 12-19 July.

Abstract

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

Enhanced RGB-D Mapping Method for Detailed 3D Modeling of Large Indoor Environments

Tang, S., Zhu, Q., Chen, W., Darwish, W., Wu, B., Hu, H., Chen, M., 2016
Conference PapersISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences III-3, 19-26. Prague Czech, 12-19 July.

Abstract

RGB-D sensors are novel sensing systems that capture RGB images along with pixel-wise depth information. Although they are widely used in various applications, RGB-D sensors have significant drawbacks with respect to 3D dense mapping of indoor environments. First, they only allow a measurement range with a limited distance (e.g., within 3 m) and a limited field of view. Second, the error of the depth measurement increases with increasing distance to the sensor. In this paper, we propose an enhanced RGB-D mapping method for detailed 3D modeling of large indoor environments by combining RGB image-based modeling and depth-based modeling. The scale ambiguity problem during the pose estimation with RGB image sequences can be resolved by integrating the information from the depth and visual information provided by the proposed system. A robust rigid-transformation recovery method is developed to register the RGB image-based and depth-based 3D models together. The proposed method is examined with two datasets collected in indoor environments for which the experimental results demonstrate the feasibility and robustness of the proposed method

Robust Low-Altitude Image Matching Based on Local Region Constraint and Feature Similarity Confidence

Chen, M., Zhu, Q., Huang, S., Hu, H., Wang, J., 2016
Conference PapersISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences III-3, 19-26. Prague Czech, 12-19 July.

Abstract

Improving the matching reliability of low-altitude images is one of the most challenging issues in recent years, particularly for images with large viewpoint variation. In this study, an approach for low-altitude remote sensing image matching that is robust to the geometric transformation caused by viewpoint change is proposed. First, multiresolution local regions are extracted from the images and each local region is normalized to a circular area based on a transformation. Second, interest points are detected and clustered into local regions. The feature area of each interest point is determined under the constraint of the local region which the point belongs to. Then, a descriptor is computed for each interest point by using the classical scale invariant feature transform (SIFT). Finally, a feature matching strategy is proposed on the basis of feature similarity confidence to obtain reliable matches. Experimental results show that the proposed method provides significant improvements in the number of correct matches compared with other traditional methods.

Orientation of Oblique Airborne Image Sets - Experiences from the ISPRS/EuroSDR Benchmark on Multi-Platform Photogrammetry

Gerke, M., Nex, F., Remondino, F., Jacobsen, K., Kremer, J., Karel, W., Hu, H., Ostrowski, W., 2016
Conference PapersThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLI-B1, 185-191. Prague Czech, 12-19 July.

Abstract

During the last decade the use of airborne multi camera systems increased significantly. The development in digital camera technology allows mounting several mid- or small-format cameras efficiently onto one platform and thus enables image capture under different angles. Those oblique images turn out to be interesting for a number of applications since lateral parts of elevated objects, like buildings or trees, are visible. However, occlusion or illumination differences might challenge image processing. From an image orientation point of view those multi-camera systems bring the advantage of a better ray intersection geometry compared to nadir-only image blocks. On the other hand, varying scale, occlusion and atmospheric influences which are difficult to model impose problems to the image matching and bundle adjustment tasks. In order to understand current limitations of image orientation approaches and the influence of different parameters such as image overlap or GCP distribution, a commonly available dataset was released. The originally captured data comprises of a state-of-the-art image block with very high overlap, but in the first stage of the so-called ISPRS/EUROSDR benchmark on multi-platform photogrammetry only a reduced set of images was released. In this paper some first results obtained with this dataset are presented. They refer to different aspects like tie point matching across the viewing directions, influence of the oblique images onto the bundle adjustment, the role of image overlap and GCP distribution. As far as the tie point matching is concerned we observed that matching of overlapping images pointing to the same cardinal direction, or between nadir and oblique views in general is quite successful. Due to the quite different perspective between images of different viewing directions the standard tie point matching, for instance based on interest points does not work well. How to address occlusion and ambiguities due to different views onto objects is clearly a non-solved research problem so far. In our experiments we also confirm that the obtainable height accuracy is better when all images are used in bundle block adjustment. This was also shown in other research before and is confirmed here. Not surprisingly, the large overlap of 80/80% provides much better object space accuracy – random errors seem to be about 2-3fold smaller compared to the 60/60% overlap. A comparison of different software approaches shows that newly emerged commercial packages, initially intended to work with small frame image blocks, do perform very well.

Stable least-squares matching for oblique images using bound constrained optimization and a robust loss function

Hu, H., Ding, Y.*, Zhu, Q., Wu, B., Xie, L., Chen, M., 2016.
Journal PapersISPRS Journal of Photogrammetry and Remote Sensing 118, 53-67.

Abstract

Least-squares matching is a standard procedure in photogrammetric applications for obtaining sub-pixel accuracies of image correspondences. However, least-squares matching has also been criticized for its instability, which is primarily reflected by the requests for the initial correspondence and favorable image quality. In image matching between oblique images, due to the blur, illumination differences and other effects, the image attributes of different views are notably different, which results in a more severe convergence problem. Aiming at improving the convergence rate and robustness of least-squares matching of oblique images, we incorporated prior geometric knowledge in the optimization process, which is reflected as the bounded constraints on the optimizing parameters that constrain the search for a solution to a reasonable region. Furthermore, to be resilient to outliers, we substituted the square loss with a robust loss function. To solve the composite problem, we reformulated the least-squares matching problem as a bound constrained optimization problem, which can be solved with bounds constrained Levenberg–Marquardt solver. Experimental results consisting of images from two different penta-view oblique camera systems confirmed that the proposed method shows guaranteed final convergences in various scenarios compared to the approximately 20–50% convergence rate of classical least-squares matching.

An asymmetric re-weighting method for the precision combined bundle adjustment of aerial oblique images

Xie, L., Hu, H.*, Wang, J., Zhu, Q., Chen, M., 2016
Journal PapersISPRS Journal of Photogrammetry and Remote Sensing 117, 92-107.

Abstract

Combined bundle adjustment is a fundamental step in the processing of massive oblique images. Traditional bundle adjustment designed for nadir images gives identical weights to different parts of image point observations made from different directions, due to the assumption that the errors in the observations follow the same Gaussian distribution. However, because of their large tilt angles, aerial oblique images have trapezoidal footprints on the ground, and their areas correspond to conspicuously different ground sample distances. The errors in different observations no longer conform to the above assumption, which leads to suboptimal bundle adjustment accuracy and restricts subsequent 3D applications. To model the distribution of the errors correctly for the combined bundle adjustment of oblique images, this paper proposes an asymmetric re-weighting method. The scale of each pixel is used to determine a re-weighting factor, and each pixel is subsequently projected onto the ground to identify another anisotropic re-weighting factor using the shape of its quadrangle. Next, these two factors are integrated into the combined bundle adjustment using asymmetric weights for the image point observations; greater weights are assigned to observations with fine resolutions, and those with coarse resolutions are penalized. This paper analyzes urban and rural images captured by three different five-angle camera systems, from both proprietary datasets and the ISPRS/EuroSDR benchmark. The results reveal that the proposed method outperforms the traditional method in both back-projected and triangulated precision by approximately 5–10% in most cases. Furthermore, the misalignments of point clouds generated by the different cameras are significantly alleviated after combined bundle adjustment.

Geometric integration of high-resolution satellite imagery and airborne LiDAR data for improved geopositioning accuracy in metropolitan areas

Wu, B., Tang, S., Zhu, Q., Tong, K., Hu, H., Li, G., 2015.
Journal PapersISPRS Journal of Photogrammetry and Remote Sensing 109, 139-151

Abstract

High-resolution satellite imagery (HRSI) and airborne light detection and ranging (LiDAR) data are widely used for deriving 3D spatial information. However, the 3D spatial information derived from them in the same area can be inconsistent. Considering HRSI and LiDAR datasets taken from metropolitan areas as a case study, this paper presents a novel approach to the geometric integration of HRSI and LiDAR data to reduce their inconsistencies and improve their geopositioning accuracy. First, the influences of HRSI’s individual rational polynomial coefficients (RPCs) on geopositioning accuracies are analyzed and the RPCs that dominate those accuracies are identified. The RPCs are then used as inputs in the geometric integration model together with the tie points identified in stereo images and LiDAR ground points. A local vertical constraint and a local horizontal constraint are also incorporated in the model to ensure vertical and horizontal consistency between the two datasets. The model improves the dominating RPCs and the ground coordinates of the LiDAR points, decreasing the inconsistencies between the two datasets and improving their geopositioning accuracy. Experiments were conducted using ZY-3 and Pleiades-1 imagery and the corresponding airborne LiDAR data in Hong Kong. The results verify that the geometric integration model effectively improves the geopositioning accuracies of both types of imagery and the LiDAR points. Furthermore, the model enables the full comparative and synergistic use of remote sensing imagery and laser scanning data collected from different platforms and sensors.

Reliable spatial relationship constrained feature point matching of oblique aerial images ( Talbert Abrams Award, Second Honorable Mention)

Hu, H., Zhu, Q., Du, Z., Zhang, Y., Ding, Y., 2015
Journal Papers Photogrammetric Engineering and Remote Sensing 81 (1), 49-58

Abstract

This paper proposes a reliable feature point matching method for oblique images using various spatial relationships and geometrical information for the problems resulted by the large view point changes, the image deformations, blurring, and other factors. Three spatial constraints are incorporated to filter possible outliers, including a cyclic angular ordering constraint, a local position constraint, and a neighborhood conserving constraint. Other ancillary geometric information, which includes the initial exterior orientation parameters that are obtained from the platform parameters and a rough DEM, are used to transform the oblique images geometrically and reduce the perspective deformations. Experiment results revealed that the proposed method is superior to the standard SIFT regarding both precision and correct matches using images obtained by the SWDC-5 system.

A Quick and Affine Invariance Matching Method for Oblique Images

Xiao, X., Guo, B., Li, D., Zhao, X., Jiang, W., Hu, H., Zhang, C., 2015
Journal Papers Acta Geodaetica et Cartographica Sinica 44 (4), 414-421

Abstract

This paper proposed a quick, affine invariance matching method for oblique images. It calculated the initial affine matrix by making full use of the two estimated camera axis orientation parameters of an oblique image, then recovered the oblique image to a rectified image by doing the inverse affine transform, and left over by the SIFT method. We used the nearest neighbor distance ratio(NNDR), normalized cross correlation(NCC) measure constraints and consistency check to get the coarse matches, then used RANSAC method to calculate the fundamental matrix and the homography matrix. And we got the matches that they were interior points when calculating the homography matrix, then calculated the average value of the matches' principal direction differences. During the matching process, we got the initial matching features by the nearest neighbor(NN) matching strategy, then used the epipolar constrains, homography constrains, NCC measure constrains and consistency check of the initial matches' principal direction differences with the calculated average value of the interior matches' principal direction differences to eliminate false matches. Experiments conducted on three pairs of typical oblique images demonstrate that our method takes about the same time as SIFT to match a pair of oblique images with a plenty of corresponding points distributed evenly and an extremely low mismatching rate.

Phase Grouping Line Extraction Algorithm Using Overlapped Partition

Wang, J., Zhu, Q., Zhang, Y., Hu, H., 2015
Journal Papers Acta Geodaetica et Cartographica Sinica 44 (7), 768-774

Abstract

Aiming at solving the problem of fracture at the discontinuities area and the challenges of line fitting in each partition, an innovative line extraction algorithm is proposed based on phase grouping using overlapped partition. The proposed algorithm adopted dual partition steps, which will generate overlapped eight partitions. Between the two steps, the middle axis in the first step coincides with the border lines in the other step. Firstly, the connected edge points that share the same phase gradients are merged into the line candidates, and fitted into line segments. Then to remedy the break lines at the border areas, the break segments in the second partition steps are refitted. The proposed algorithm is robust and does not need any parameter tuning. Experiments with various datasets have confirmed that the method is not only capable of handling the linear features, but also powerful enough in handling the curve features.

Contour Cluster Shape Analysis for Building Damage Detection from Post-earthquake Airborne LiDAR

He, M., Zhu, Q., Du, Z., Zhang, Y., Hu, H., Lin, Y., Qi, H., 2015
Journal Papers Acta Geodaetica et Cartographica Sinica 44 (4), 407-413

Abstract

Detection of the damaged building is the obligatory step prior to evaluate earthquake casualty and economic losses. It's very difficult to detect damaged buildings accurately based on the assumption that intact roofs appear in laser data as large planar segments whereas collapsed roofs are characterized by many small segments. This paper presents a contour cluster shape similarity analysis algorithm for reliable building damage detection from the post-earthquake airborne LiDAR point cloud. First we evaluate the entropies of shape similarities between all the combinations of two contour lines within a building cluster, which quantitatively describe the shape diversity. Then the maximum entropy model is employed to divide all the clusters into intact and damaged classes. The tests on the LiDAR data at El Mayor-Cucapah earthquake rupture prove the accuracy and reliability of the proposed method.

An adaptive surface filter for airborne laser scanning point clouds by means of regularization and bending energy

Hu, H., Ding, Y., Zhu, Q., Wu, B., Lin, H., Du, Z., Zhang, Y., Zhang, Y., 2014
Journal Papers ISPRS Journal of Photogrammetry and Remote Sensing 92, 98-111

Abstract

Descriptions of the bending energy and the transformed compensation value generated during interpolation of the corresponding raster surface. (a) The interpolated raster surface, (b) the generated bending energy as a by-product of the TPS interpolation and (c) the transformed bend_gain by piece-wise linear interpolation from the bending energy raster given a upper bound of 0.3 m.

The filtering of point clouds is a ubiquitous task in the processing of airborne laser scanning (ALS) data; however, such filtering processes are difficult because of the complex configuration of the terrain features. The classical filtering algorithms rely on the cautious tuning of parameters to handle various landforms. To address the challenge posed by the bundling of different terrain features into a single dataset and to surmount the sensitivity of the parameters, in this study, we propose an adaptive surface filter (ASF) for the classification of ALS point clouds. Based on the principle that the threshold should vary in accordance to the terrain smoothness, the ASF embeds bending energy, which quantitatively depicts the local terrain structure to self-adapt the filter threshold automatically. The ASF employs a step factor to control the data pyramid scheme in which the processing window sizes are reduced progressively, and the ASF gradually interpolates thin plate spline surfaces toward the ground with regularization to handle noise. Using the progressive densification strategy, regularization and self-adaption, both performance improvement and resilience to parameter tuning are achieved. When tested against the benchmark datasets provided by ISPRS, the ASF performs the best in comparison with all other filtering methods, yielding an average total error of 2.85% when optimized and 3.67% when using the same parameter set.

Integration of Chang'E-2 imagery and LRO laser altimeter data with a combined block adjustment for precision lunar topographic modeling

Wu, B., Hu, H.*, Guo, J., 2014
Journal Papers Earth and Planetary Science Letters 391, 1-15

Highlights

  • Cross-mission and cross-sensor data integration for precision lunar topographic modeling.
  • Image residuals reduced from tens of pixels before the adjustment to sub-pixel level after adjustment.
  • After block adjustment the Chang'E-2 DEM showed good consistency with the LOLA DEM.

Abstract

Lunar topographic information is essential for lunar scientific investigations and exploration missions. Lunar orbiter imagery and laser altimeter data are two major data sources for lunar topographic modeling. Most previous studies have processed the imagery and laser altimeter data separately for lunar topographic modeling, and there are usually inconsistencies between the derived lunar topographic models. This paper presents a novel combined block adjustment approach to integrate multiple strips of the Chinese Chang'E-2 imagery and NASA's Lunar Reconnaissance Orbiter (LRO) Laser Altimeter (LOLA) data for precision lunar topographic modeling. The participants of the combined block adjustment include the orientation parameters of the Chang'E-2 images, the intra-strip tie points derived from the Chang'E-2 stereo images of the same orbit, the inter-strip tie points derived from the overlapping area of two neighbor Chang'E-2 image strips, and the LOLA points. Two constraints are incorporated into the combined block adjustment including a local surface constraint and an orbit height constraint, which are specifically designed to remedy the large inconsistencies between the Chang'E-2 and LOLA data sets. The output of the combined block adjustment is the improved orientation parameters of the Chang'E-2 images and ground coordinates of the LOLA points, from which precision lunar topographic models can be generated. The performance of the developed approach was evaluated using the Chang'E-2 imagery and LOLA data in the Sinus Iridum area and the Apollo 15 landing area. The experimental results revealed that the mean absolute image residuals between the Chang'E-2 image strips were drastically reduced from tens of pixels before the adjustment to sub-pixel level after adjustment. Digital elevation models (DEMs) with 20 m resolution were generated using the Chang'E-2 imagery after the combined block adjustment. Comparison of the Chang'E-2 DEM with the LOLA DEM showed a good level of consistency. The developed combined block adjustment approach is of significance for the full comparative and synergistic use of lunar topographic data sets from different sensors and different missions.

Integration and Coregistration of Multisource Lunar Topographic Data Sets for Synergistic Use

Wu, B., Guo, J., Hu, H., 2014.
Book Chapters In: Jin, S. (Eds.), Planetary Geodesy and Remote Sensing. Taylor & Francis Group/CRC Press, Boca Raton, FL, USA, pp. 99-120

3D Building Facades and Roofs Objects Extraction from Pixel Height Map

Huang, M., Du, Z., Zhu, Q., Zhang, Y., Hu, H., 2014
Journal Papers Geomatics and Information Science of Wuhan University 39 (10), 1221-1224

Abstract

Aiming at the problem of extracting 3D building facades and roofs automatically, a 3D building extraction algorithm from pixel height map, which is a product of oblique airborne imagery, is therefore designed and implemented preliminarily. Compared with no or gradual change on the ground or roofs, vertical facade planes show large height gradients. So, this paper take advantage of the height gradient feature to extract facades, at the same time use the maximum entropy threshold feature to extract roof. Oblique images used in this research were collected by Beijing Geo-Vision Tech.Co.,Ltd. with the SWDC-5 Digital aerial photographic system. The results show that it is an effective method to combine the height gradient feature with maximum entropy threshold feature for extracting buildings from pixel height map.

A flexible method for zoom lens calibration and modeling using a planar checkerboard

Wu, B., Hu, H., Zhu, Q., Zhang, Y., 2013.
Journal PapersPhotogrammetric Engineering and Remote Sensing 79 (6), 555-571

Abstract

This paper presents a fl exible method for zoom lens calibration and modeling using a planar checkerboard. The method includes the following four steps. First, the principal point of the zoom-lens camera is determined by a focus-of-expansion approach. Second, the infl uences of focus changes on the principal distance are modeled by a scale parameter. Third, checkerboard images taken at varying object distances with convergent image geometry are used for camera calibration. Finally, the variations of the calibration parameters with respect to the various zoom and focus settings are modeled using polynomials. Three different types of lens are examined in this study. Experimental analyses show that high precision calibration results can be expected from the developed approach. The relative measurement accuracy (accuracy normalized with object distance) using the calibrated zoom-lens camera model ranges from 1:5 000 to 1:25 000. The developed method is of signifi cance to facilitate the use of zoom-lens camera systems in various applications such as robotic exploration, hazard monitoring, traffi c monitoring, and security surveillance.

Co-registration of lunar topographic models derived from Chang’E-1, SELENE, and LRO laser altimeter data based on a novel surface matching method

Wu, B., Guo, J., Hu, H., Li, Z., Chen, Y., 2013
Journal Papers Earth and Planetary Science Letters 364, 68-84.

Abstract

Various lunar digital topographic models (DTMs) have been generated from the data collected from earlier and recent lunar missions. There are usually inconsistencies among them due to differences in sensor configurations, data acquisition periods, and production techniques. To obtain maximum value for science and exploration, the multi-source lunar topographic datasets must be co-registered in a common reference frame. Only such an effort will ensure the proper calibration, registration, and analysis of the datasets, which in turn will permit the full comparative and synergistic use of them. This study presents a multi-feature-based surface matching method for the co-registration of multiple lunar DTMs that incorporates feature points, lines, and surface patches in surface matching to guarantee robust surface correspondence. A combined adjustment model is developed for the determination of seven transformation parameters (one scale factor, three rotations, and three translations), from which the multiple DTMs could be co-registered. The lunar DTMs derived from the Chang’E-1, SELENE, and LRO laser altimeter data in the Apollo 15 landing area and the Sinus Iridum area are examined in this study. Small offsets were found among the Chang’E-1, SELENE, and LRO DTMs. Through experimental analysis, the developed multi-feature-based method was proven able to effectively co-register multiple lunar DTMs. The performances of the multi-feature-based surface matching method were compared with the point-based method, and the former was proven to be superior to the latter.

A Multi-Level Cache Approach for Realtime Visualization of Massive 3D GIS Data

Li, X., Xu, W., Zhu, Q., Hu, J., Hu, H., Zhang, Y., 2012
Journal Papers International Journal of 3-D Information Modeling 1 (3), 37-48.

Chengdu

Chengdu is a quite nice place with delicious food and warm people.

Here, we have a group working on virtual geographic enviroment. Our recent interests are focused on turing oblique images into 3D models, both automatically and interactively.

Hong Kong

Hong Kong is a fantastic modern city.

Here, we have a group working on planetary mapping, majorly for Moon and Mars.