How to use Multi-band Blender in opencv

Question

I want to blend two images using multiband blending but I am not clear to the input parameter of this function:

void detail::Blender::prepare(const std::vector<Point>& corners, const std::vector<Size>& sizes)

In my case ,I just input two warped images with black gap, and with masks all white.(forgive me can not add pictures...)

And I set the two corners (0.0,0.0),because the warped images has been registered.

but my result is not good enough.with obvious seam in the result

can someone tell me why?How can I solve this problem?

Answer 1

I'm not sure what do you mean when you say "my result is not good enough". It's better to watch that result, but I'll try to guess. My main part of code, which makes panorama, looks like this:

void makePanorama(Rect bounding_box, vector<Mat> images, vector<Mat> homographies, vector<vector<Point>> corners) {
  detail::MultiBandBlender blender;
  blender.prepare(bounding_box);

  Mat mask, bigImage, curImage;
  for (int i = 0; i < (int)images.size(); ++i) {
    warpPerspective(images[i], curImage, homographies[i],
                bounding_box.size(), INTER_LINEAR, ORDER_TRANSPARENT);

    mask = makeMask(curImage.size(), corners[i], homographies[i]);
    blender.feed(curImage.clone(), mask, Point(0, 0));
  }

  blender.blend(bigImage, mask);
  bigImage.convertTo(bigImage, (bigImage.type() / 8) * 8);
  imshow("Result", bigImage);
  waitKey();
}

So, prepare blender and then loop: warp image, make the mask after warped image and feed blender. At the end, turn this blender on and that's all. I met two problems, which influence on my result badly. May be you have one of them or both.

The first is type. My images had CV_16SC3, and after blending you need to convert blended image type into unsigned one. Like this

  bigImage.convertTo(bigImage, (bigImage.type() / 8) * 8);

If you not, the result image would be gray.

The second is borders. In the beginning, my function makeMask was calculating non-black area of warped images. As a result, the one could see borders of the warped images on the blended image. The solution is to make mask smaller than non-black warped image area. So, my function makeMask is looks like this:

Mat makeMask(Size sz, vector<Point2f> imageCorners, Mat homorgaphy) {
  Scalar white(255, 255, 255);
  Mat mask = Mat::zeros(sz, CV_8U);
  Point2f innerPoint;
  vector<Point2f> transformedCorners(4);

  perspectiveTransform(imageCorners, transformedCorners, homorgaphy);
  // Calculate inner point
  for (auto& point : transformedCorners)
      innerPoint += point;
  innerPoint.x /= 4;
  innerPoint.y /= 4;

  // Make indent for each corner
  vector<Point> corners;
  for (int ind = 0; ind < 4; ++ind) {
    Point2f direction = innerPoint - transformedCorners[ind];
    double normOfDirection = norm(direction);
    corners[ind].x += settings.indent * direction.x / normOfDirection;
    corners[ind].y += settings.indent * direction.y / normOfDirection;
  }

  // Draw borders
  Point prevPoint = corners[3];
  for (auto& point : corners) {
    line(mask, prevPoint, point, white);
    prevPoint = point;
  }

  // Fill with white
  floodFill(mask, innerPoint, white);
  return mask;
}

I took this pieces of code from my real code, so I could possibly forget to specify something. But I hope, the idea of how to work with MultiBandBlender is clear.