For scoring and analysis, DXOMARK engineers capture and evaluate more than 2,600 test images both in controlled lab environments and in outdoor, indoor and low-light natural scenes, using the camera’s default settings. The photo protocol is designed to take into account the main use cases and is based on typical shooting scenarios, such as portraits, family, and landscape photography. The evaluation is performed by visually inspecting images against a reference of natural scenes, and by running objective measurements on images of charts captured in the lab under different lighting conditions from 1 to 1,000+ lux and color temperatures from 2,300K to 6,500K.

Exposure and color are the key attributes for technically good pictures. For exposure, the main attribute evaluated is the brightness of the main subject through various use cases such as landscape, portrait, or still life. Other factors evaluated are the contrast and the dynamic range, eg. the ability to render visible details in both bright and dark areas of the image. Repeatability is also important because it demonstrates the camera’s ability to provide the same rendering when shooting several images of the same scene.
For color, the image quality attributes analyzed are skin-tone rendering, white balance, color shading, and repeatability. For color and skin tone rendering, we penalize unnatural colors but we respect a manufacturer’s choice of color signature.

Autofocus tests concentrate on focus accuracy, focus repeatability, shooting time delay, and depth of field. Shooting delay is the difference between the time the user presses the capture button and the time the image is actually taken. It includes focusing speed and the capability of the device to capture images at the right time, what is called ‘zero shutter lag’ capability. Even if a shallow depth of field can be pleasant for a single subject portrait or close-up shot, it can also be a problem in some specific conditions such as group portraits; Both situations are tested. Focus accuracy is also evaluated in all the real-life images taken, from infinity to close-up objects and in low light to outdoor conditions.

Texture tests analyze the level of details and the texture of subjects in the images taken in the lab as well as in real-life scenarios. For natural shots, particular attention is paid to the level of details in the bright and dark areas of the image. Objective measurements are performed on chart images taken in various lighting conditions from 1 to 1000 lux and different kinds of dynamic range conditions. The charts used are the proprietary DXOMARK chart (DMC) and the Dead Leaves chart.

Noise tests analyze various attributes of noise such as intensity, chromaticity, grain, structure on real-life images as well as images of charts taken in the lab. For natural images, particular attention is paid to the noise on faces, landscapes, but also on dark areas and high dynamic range conditions. Noise on moving objects is also evaluated on natural images. Objective measurements are performed on images of charts taken in various conditions from 1 to 1000 lux and different kinds of dynamic range conditions. The chart used is the Dead Leaves chart and the standardized measurement such as Visual Noise derived from ISO 15739.

The artifacts evaluation looks at lens shading, chromatic aberrations, geometrical distortion, edges ringing, halos, ghosting, quantization, unexpected color hue shifts, among others type of possible unnatural effects on photos. The more severe and the more frequent the artifact, the higher the point deduction on the score. The main artifacts observed and corresponding point loss are listed below.

Bokeh is tested in one dedicated mode, usually portrait or aperture mode, and analyzed by visually inspecting all the images captured in the lab and in natural conditions. The goal is to reproduce portrait photography comparable to one taken with a DLSR and a wide aperture. The main image quality attributes paid attention to are depth estimation, artifacts, blur gradient, and the shape of the bokeh blur spotlights. Portrait image quality attributes (exposure, color, texture) are also taken into account.

Preview tests analyze the image quality of the camera app’s preview of the image, with particular attention paid to the difference between the capture and the preview, especially regarding dynamic range and the application of the bokeh effect. Also evaluated is the smoothness of the exposure, color and focus adaptation when zooming from the minimal to the maximal zoom factor available. The preview frame rate is measured using the LED Universal Timer.

DXOMARK engineers capture and evaluate over 400 test images in controlled lab environments and in outdoor, indoor, and low-light natural scenes, using the camera’s default settings and pinch zoom at various zoom factors from ultra wide to very long-range zoom. The evaluation is performed by visually inspecting the images against a reference of natural scenes, and by running objective measurements of chart mages captured in the lab under different conditions from 20 to 1000 lux and color temperatures from 2300K to 6500K.

These tests analyze the performance of the ultra-wide camera at several focal lengths from 12 mm to 20 mm. All image quality attributes are evaluated, with particular attention paid to such artifacts as chromatic aberrations, lens softness, and distortion. Pictures below are an extract of tested scenes.

All image quality attributes are evaluated at focal lengths from approximately 40 mm to 300 mm, with particular attention paid to texture and detail. The score is derived from a number of objective measurements in the lab and perceptual analysis of real-life images.