Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenCV does not respect exif orientation in PNG file #16579

Open
codars opened this issue Feb 13, 2020 · 8 comments
Open

OpenCV does not respect exif orientation in PNG file #16579

codars opened this issue Feb 13, 2020 · 8 comments

Comments

@codars
Copy link

@codars codars commented Feb 13, 2020

System information (version)
  • OpenCV => 4.2
  • Operating System / Platform => Windows 64 Bit
  • Compiler => Visual Studio 2015
Detailed description

PNG format starts to support EXIF information since 2017, but somehow using IMREAD_COLOR to read a PNG file does not respect the orientation

Attached Capture.png and Capture.jpg are two images both with same EXIF orientation information, try both and you can see the difference.

Steps to reproduce
    // C++ code example
    cv::Mat cvImgColor = cv::imread("Capture.png", cv::IMREAD_COLOR);
    // Expected: cols = 333, rows = 280
    // Actual: cols = 280, rows = 333

tempsnip

Capture

Capture

@Infinity8sailor
Copy link

@Infinity8sailor Infinity8sailor commented Feb 24, 2020

Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.

import numpy as np
import cv2
img1=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2.jpg",1)
img2=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2.jpg",-1)
img3=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2.jpg",128)
np.shape(img1)
(183, 275, 3)
np.shape(img2)
(183, 275, 3)
np.shape(img3)
(183, 275)
img1=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2p.jpg",1)
img2=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2p.jpg",-1)
img3=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\sand_2p.jpg",128)
np.shape(img1)
(275, 183, 3)
np.shape(img2)
(183, 275, 3)
np.shape(img3)
(183, 275)
img1=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_portrait.png",1)
img2=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_portrait.png",-1)
img3=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_portrait.png",128)
np.shape(img1)
(2340, 1080, 3)
np.shape(img2)
(2340, 1080, 3)
np.shape(img3)
(2340, 1080)
img1=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_land_port.png",1)
img2=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_land_port.png",-1)
img3=cv2.imread(r"C:\Users\vaibhav\vs\cv\issue\png\screenshots\youtube_land_port.png",128)
np.shape(img1)
(1080, 2340, 3)
np.shape(img2)
(1080, 2340, 3) **# expected (2340, 1080, 3) **
np.shape(img3)
(1080, 2340) **# expected (2340, 1080) **

  • flags 1(IMREAD_UNCHANGED) ,128(IMREAD_IGNORE_ORIENTATION) works fine with jpg but not with PNG
@Infinity8sailor
Copy link

@Infinity8sailor Infinity8sailor commented Feb 25, 2020

hey i am not getting that affected file
can you (@alalek )please help where i should be searching that file???
this is my very first issue/bug...

@codars
Copy link
Author

@codars codars commented Feb 25, 2020

@Infinity8sailor Hi, the affected file is the second png in my description, the 2nd image and 3rd image are png/jpg with exif information

@pandeykriti
Copy link

@pandeykriti pandeykriti commented Mar 9, 2020

Is this issue still open!?(@alalek )

@OsamaNabih
Copy link

@OsamaNabih OsamaNabih commented Mar 18, 2020

I've been working on this issue for a few days now. I'm extremely familiar with both the exif format now and the code that parses it. The problem is, the current code assumes the exif format found inside the JPG images only. However, PNG's format is different from JPG's, not only can it be found in any arbitrary place between IHDR and IEND, moreover, some of the tags that exist in JPG EXIF chunk were discarded, most notably the APP0 and APP1.
I am still trying to figure out how to best adjust the code, I don't want to mess with the gfrmt_png files, I'd prefer the EXIF part was handled after the actual reading of the image, similar to what is currently done for JPEG. However the current ExifReader class is built around the JPEG format.
Any guidance as to how to approach this will be helpful.

@alalek
Copy link
Contributor

@alalek alalek commented Mar 18, 2020

@OsamaNabih Thank you for looking on this!

the current code assumes the exif format found inside the JPG images only

Looks like you are right.

Not sure that we can properly adapt current ExifReader for other formats, at least in a reliable way (we must avoid false positives here).

Perhaps we should extend BaseImageDecoder to support reading of EXIF data (probably as part of .readHeader()).

Format-specific reader should be able to properly find EXIF chunk of data (own code or through underlying I/O library like png_get_eXIf_1@libpng).

Some common code (if format is the same) can be used to decode EXIF buffer. We can try to reuse some part of current ExifReader as ExifDecoder for that.

@OsamaNabih
Copy link

@OsamaNabih OsamaNabih commented Mar 18, 2020

The tiff reading parts (most of the functions in exif.hpp/cpp) may be reused, however, for the functionality pointed to in this issue (using imread), it only cares about the orientation tag.
Another question I have is that as far as I understand the EXIF format in the PNG specification doesn't have a length field, does this mean we'll be basically reading all the file to search for the exif chunk then continue until there's nothing left to read? Doesn't sound great.
For the BaseImageDecoder part, do you mean extending it in grfmt_base or grfmt_png?

@OsamaNabih
Copy link

@OsamaNabih OsamaNabih commented Mar 19, 2020

My bad about not including the length, while it's true it doesn't give bytes to read like the JPEG, there's a number of tiff headers to read, so we won't be going on indefinitely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.