python裁剪图片边缘,使用Python自动裁剪图像以提取内部黑色边框ROI

I’ve been looking into OpenCV and Pillow (and ImageMagick outside of Python, especially Fred's Image Magick Scripts) to achieve the following:

Auto identification of an inner black border in scanned Images and cropping of the images to this said border. Here is a blacked out example Image, the first is the "original" and the second one with a red highlight around the black border is what I am looking to achieve:

The problem is that the border is not on the outside of the images and the scans differ greatly in quality, meaning the border is never on the same spot and it’s not possible to crop by means of pixels.

Edit: I’m looking for a way to crop the image only keeping everything inside the black border (what is blurred right now)

I’m looking for help on how a) if it is possible to do such a cropping and b) how to do it preferably with Python.

Thanks!

解决方案

Here is a pretty simple way to do that in Imagemagick.

Get the center coordinates

Clone the image and do the following on the clone

Threshold the image so that the inside of the black lines is white.

(If necessary use -connected-components to merge smaller black features into the white in the center)

Apply some morphology open to make sure that the black lines are continuous

Floodfill the image with red starting in the center

Convert non-red to black and red to white

Put the processed clone into the alpha channel of the input

Input:

center=$(convert img.jpg -format "%[fx:w/2],%[fx:h/2]\n" info:)

convert img.jpg \

\( +clone -auto-level -threshold 35% \

-morphology open disk:5 \

-fill red -draw "color $center floodfill" -alpha off \

-fill black +opaque red -fill white -opaque red \) \

-alpha off -compose copy_opacity -composite result.png

Here is Python Wand code that is the equivalent to the above:

#!/bin/python3.7

from wand.image import Image

from wand.drawing import Drawing

from wand.color import Color

from wand.display import display

with Image(filename='black_rect.jpg') as img:

with img.clone() as copied:

copied.auto_level()

copied.threshold(threshold=0.35)

copied.morphology(method='open', kernel='disk:5')

centx=round(0.5*copied.width)

centy=round(0.5*copied.height)

with Drawing() as draw:

draw.fill_color='red'

draw.color(x=centx, y=centy, paint_method='floodfill')

draw(copied)

copied.opaque_paint(target='red', fill='black', fuzz=0.0, invert=True)

copied.opaque_paint(target='red', fill='white', fuzz=0.0, invert=False)

display(copied)

copied.alpha_channel = 'copy'

img.composite(copied, left=0, top=0, operator='copy_alpha')

img.format='png'

display(img)

img.save(filename='black_rect_interior.png')

For OpenCV, I would suggest that the following processing could be one way to do it. Sorry, I am not proficient with OpenCV

Threshold the image so that the inside of the black lines is white.

Apply some morphology open to make sure that the black lines are continuous

Get the contours of the white regions.

Get the largest interior contour and fill the inside with white

Put that result into the alpha channel of the input

ADDITION:

For those interested, here is a longer method that would be conducive to perspective rectification. I do something similar to what nathancy has done, but in Imagemagick.

First, threshold the image and do morphology open to be sure the black lines are continuous.

Then do connected components to get the ID number of the largest white region

Then extract that region

id=$(convert img.jpg -auto-level -threshold 35% \

-morphology open disk:5 -type bilevel \

-define connected-components:mean-color=true \

-define connected-components:verbose=true \

-connected-components 8 null: | grep "gray(255)" | head -n 1 | awk '{print $1}' | sed 's/[:]*$//')

echo $id

convert img.jpg -auto-level -threshold 35% \

-morphology open disk:5 -type bilevel \

-define connected-components:mean-color=true \

-define connected-components:keep=$id \

-connected-components 8 \

-alpha extract -morphology erode disk:5 \

region.png

Now do Canny edge detection and hough line transform. Here I save the canny image, the hough lines as red lines and the lines overlaid on the image and the line information, which is saved in the .mvg file.

convert region.png \

\( +clone -canny 0x1+10%+30% +write region_canny.png \

-background none -fill red -stroke red -strokewidth 2 \

-hough-lines 9x9+400 +write region_lines.png +write lines.mvg \) \

-compose over -composite region_hough.png

convert region_lines.png -alpha extract region_bw_lines.png

# Hough line transform: 9x9+400

viewbox 0 0 2000 2829

# x1,y1 x2,y2 # count angle distance

line 0,202.862 2000,272.704 # 763 92 824

line 204.881,0 106.09,2829 # 990 2 1156

line 1783.84,0 1685.05,2829 # 450 2 2734

line 0,2620.34 2000,2690.18 # 604 92 3240

Next I use a script that I wrote to do corner detection. Here I use the Harris detector.

corners=$(corners -m harris -t 40 -d 5 -p yes region_bw_lines.png region_bw_lines_corners.png)

echo "$corners"

pt=1 coords=195.8,207.8

pt=2 coords=1772.8,262.8

pt=3 coords=111.5,2622.5

pt=4 coords=1688.5,2677.5

Next I extract and sort just the corners in clockwise fashion. The following is some code I wrote that I converted from here

list=$(echo "$corners" | sed -n 's/^.*=\(.*\)$/\1/p' | tr "\n" " " | sed 's/[ ]*$//' )

echo "$list"

195.8,207.8 1772.8,262.8 111.5,2622.5 1688.5,2677.5

# sort on x

xlist=`echo "$list" | tr " " "\n" | sort -n -t "," -k1,1`

leftmost=`echo "$xlist" | head -n 2`

rightmost=`echo "$xlist" | tail -n +3`

rightmost1=`echo "$rightmost" | head -n 1`

rightmost2=`echo "$rightmost" | tail -n +2`

# sort leftmost on y

leftmost2=`echo "$leftmost" | sort -n -t "," -k2,2`

topleft=`echo "$leftmost2" | head -n 1`

btmleft=`echo "$leftmost2" | tail -n +2`

# get distance from topleft to rightmost1 and rightmost2; largest is bottom right

topleftx=`echo "$topleft" | cut -d, -f1`

toplefty=`echo "$topleft" | cut -d, -f2`

rightmost1x=`echo "$rightmost1" | cut -d, -f1`

rightmost1y=`echo "$rightmost1" | cut -d, -f2`

rightmost2x=`echo "$rightmost2" | cut -d, -f1`

rightmost2y=`echo "$rightmost2" | cut -d, -f2`

dist1=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost1x),($toplefty-$rightmost1y))]" info:`

dist2=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost2x),($toplefty-$rightmost2y))]" info:`

test=`convert xc: -format "%[fx:$dist1>$dist2?1:0]" info:`

if [ $test -eq 1 ]; then

btmright=$rightmost1

topright=$rightmost2

else

btmright=$rightmost2

topright=$rightmost1

fi

sort_corners="$topleft $topright $btmright $btmleft"

echo $sort_corners

195.8,207.8 1772.8,262.8 1688.5,2677.5 111.5,2622.5

Finally, I use the corner coordinates to draw a white filled polygon on a black background and put that result into the alpha channel of the input image.

convert img.jpg \

\( +clone -fill black -colorize 100 \

-fill white -draw "polygon $sort_corners" \) \

-alpha off -compose copy_opacity -composite result.png