JX3fExtract Library and Command Line Interface
JX3fExtract is a set of Java modules which provides the following features:
- extracting of processed for preview images and metadata embedded in Foveon X3F files ("FOVb")
- extraction of X3F data from X3I files ("FOVi")
- a command line interface to the above
The minimum Java source code level to compile this library is 11.
Module dependencies
The following diagram shows the dependencies between the modules (module names are abbreviated by omitting the common domain name prefix):
![[Module dependencies for JX3fExtract]](ModuleDependencies.svg)
Library usage
The easiest way to use the library is to call one of the overloaded
FovDataFactory.parseOrRecover(…)
methods.
For a more advanced usage see their implementation in the source code. Two of the implementations are shown here as an example – the following is for file access:
public static <R> Optional<R> parseOrRecover(Path file, boolean readFirstFOVbOnly, BiFunction<RandomAccessData, FovDataType, Optional<R>> resultHandler) throws RandomAccessDataException { FovDataType fovData; try (RandomAccessData randomAccess = RandomAccessDataFactory.createInstance(file)) { fovData = parse(randomAccess, readFirstFOVbOnly); if (!fovData.isParsedSuccessfully() && fovData.isRecoverable()) { try (RandomAccessData stream = RandomAccessDataFactory .createInstance(file.toFile(), AccessType.STREAM)) { fovData = recover(stream); } } return (resultHandler != null) ? resultHandler.apply(randomAccess, fovData) : Optional.empty(); } }
and the following is for in memory data given as a byte array:
public static <R> Optional<R> parseOrRecover(byte[] bytes, boolean readFirstFOVbOnly, BiFunction<RandomAccessData, FovDataType, Optional<R>> resultHandler) throws RandomAccessDataException { FovDataType fovData; try (RandomAccessData randomAccess = RandomAccessDataFactory.createInstance(bytes)) { fovData = parse(randomAccess, readFirstFOVbOnly); if (!fovData.isParsedSuccessfully() && fovData.isRecoverable()) { randomAccess.setCurrentOffset(0); fovData = recover(randomAccess); } return (resultHandler != null) ? resultHandler.apply(randomAccess, fovData) : Optional.empty(); } }
For a more comprehensive, working example see the source code of the
command line module de.bernd_michaely.x3f.extractor.cli .
Upgrading from version 1.0
Module de.bernd_michaely.x3f.types essentially provides – together with its dependencies –
the functionality of version 1.0 of this library.
It is still possible to use this module on its own (package names have changed because of the Java9 modularisation, they would have been to quirky otherwise, so you might have to adapt some import statements in existing code). This is the easiest upgrade path, if you e.g. just want to modularize an existing working application.
Nonetheless it is strongly recommended to use the new
de.bernd_michaely.x3i.types module
which encapsulates and extends the functionality of the former module.
(In particular the performance of the recovery process has been dramatically improved.)
Command line usage
The *.jar distribution of the command line interface (without a linked Java Runtime Environment)
can be run as shown with the provided JX3fExtract bash script:
#!/bin/bash VM_OPTIONS= #VM_OPTIONS=--show-version # location of *.jar modules: DIR=`dirname $0` # (in this case the *.jar modules reside in the same directory as this script) # alternative layout: # script in /usr/local/bin and *.jar modules in: #DIR=/usr/local/share/java/jx3fextract $JAVA_HOME/bin/java $VM_OPTIONS -p "$DIR" -m de.bernd_michaely.x3f.extractor.cli "$@"
For help on command line usage simply run the application without arguments or
one of the --help or --help-extended arguments.
The output of the latter is shown here:
Utility for extraction of processed for preview images and
metadata from X3F and X3I files, including automatic recovery.
Library version : 2.2
Supported X3F data version : 4.x
X3I support : yes
OPTIONS:
-c --color switch colorized output on (default)
-x --extract same as -j and -f
-j --extract-jpg extract embedded jpg from X3F or X3I <file> to <file.jpg>
-f --extract-x3f extract embedded X3F data from X3I files
-F --first-only for X3I files, with options -j, -m, -f,
use the first embedded X3F entry only
-h --help print this help to stdout
-H --help-extended print extended help with examples
-m --metadata print metadata from X3F and X3I file to stdout
-C --no-color switch colorized output off
-o --overwrite overwrite existing files
-s --subdir write embedded data from X3I to a subdirectory
named after the X3I files basename
-S --subdir-single write embedded data from all X3I to one single separate directory
with the name given as parameter (relative or absolute)
-d --target-dir target base directory for extracted files,
default is the current working directory
-v --verbose be verbose
-V --version print version information
For input files preceded with a relative path, this path is replicated in the target directory.
For input files preceded with an absolute path, this path is ignored.
EXAMPLES:
Print metadata for SomeFile.X3F to stdout:
> JX3fExtract -m SomeFile.X3F
or
> JX3fExtract --metadata SomeFile.X3F
Extract all data embedded in X3F and X3I files in current directory:
> JX3fExtract -vx *
Extract all X3F data embedded in X3I files in current directory,
for each X3I file into a separate subdirectory:
> JX3fExtract -vfs *.X3I
Use the find command to extract JPGs from all X3F files
found recursively in all subdirectories to 'outputdir':
> find . -type f -name '*.X3F' -exec JX3fExtract -vjd outputdir '{}' +
(Note that using the form of the -exec action closed with '+' may be
significantly faster than the form closed with ';' .)
Camera model support
The library supports the following camera models with the denoted features:
| Camera model | Processed for preview image | Thumbnail | Header information | X3F properties |
|---|---|---|---|---|
| Polaroid x530 | JPG | Uncompressed RGB (supported, converted to JPG) | Supported | Supported |
| Sigma SD9, SD10 | Proprietary data format, currently unsupported | Uncompressed RGB (supported, converted to JPG) | Supported | Supported |
| Sigma SD14, SD15, DP1, DP2, DP1s, DP2s, DP1x, DP2x | JPG | JPG | Supported | Supported |
| Sigma Merrill generation | JPG | JPG | Partially supported (format 2.x incomplete, format 3.x unsupported) | Supported |
| Sigma Quattro generation | JPG | (Not applicable, access possible to the thumbnail embedded in the JPG's Exif metadata) | Partially supported | (Not applicable) |
Furthermore it is possible to extract the X3F data embedded in the X3I file format.
Note that for the processed for preview images in JPG format
there is the RadInputStream
adapter class which provides an input stream interface
to a random access data source.
This allows for efficient access to the Exif metadata embedded in the
JPG files, e.g. by third party libraries.
Metadata output example
Reading file »SDIM5300.X3F« …
file type : FOVb
header version : 2.2
is reader compatible ? : yes
has extended 2.1 header ? : yes
Unique Identifier : 30 31 30 30 39 36 37 38 0D 6B 17 52 32 31 34 66
Mark bits : 00000000 00000000 00000000 00000000
: 76543210 76543210 76543210 76543210
Image columns : 2640
Image rows : 1760
Rotation : 0 degrees
White balance label : »Auto«
Extended header data [ 1] : Type 1 : Exposure adjust : -0.300
Extended header data [ 2] : Type 2 : Contrast adjust : 0.500
Extended header data [ 3] : Type 3 : Shadow adjust : -0.100
Extended header data [ 4] : Type 4 : Highlight adjust : 0.800
Extended header data [ 5] : Type 5 : Saturation adjust : 0.000
Extended header data [ 6] : Type 6 : Sharpness adjust : 0.000
Extended header data [ 7] : Type 10 : X3 Fill Light adjust : 0.300
Extended header data [ 8] : Type 7 : Color adjust red : 1.000
Extended header data [ 9] : Type 8 : Color adjust green : 1.000
Extended header data [10] : Type 9 : Color adjust blue : 1.000
parser in recovery mode ? : no
–––––––––––––––––
DIRECTORY SECTION
–––––––––––––––––
Size of data source : 0x00EF4F00
Directory Pointer @ : 0x00EF4EFC
Directory section @ : 0x00EF4EB4
Directory version : 2.0
Number of dir entries : 5
Directory entry # 1 : X3fDirectoryEntry [@ 0x000000E8 + 0x0014BACC = 0x0014BBB4 (IMA2)]
Directory entry # 2 : X3fDirectoryEntry [@ 0x0014BBB4 + 0x00068B84 = 0x001B4738 (CAMF)]
Directory entry # 3 : X3fDirectoryEntry [@ 0x001B4738 + 0x00000454 = 0x001B4B8C (PROP)]
Directory entry # 4 : X3fDirectoryEntry [@ 0x001B4B8C + 0x00D3C13C = 0x00EF0CC8 (IMA2)]
Directory entry # 5 : X3fDirectoryEntry [@ 0x00EF0CC8 + 0x000041EC = 0x00EF4EB4 (IMA2)]
–––––––––––––
PROPERTY LIST
–––––––––––––
Auto exposure mode [»AEMODE«] = 8-segment [»8«]
Auto focus mode [»AFMODE«] = single [»AF-S«]
Aperture value (exact) [»APERTURE«] = 8.00000 [»8.00000«]
Aperture display [»AP_DESC«] = 8.0 [»8.0«]
Auto bracketing index [»BRACKET«] = [» «]
Burst mode index [»BURST«] = 0 [»0«]
Camera manufacturer [»CAMMANUF«] = SIGMA [»SIGMA«]
Camera model [»CAMMODEL«] = SIGMA DP1 [»SIGMA DP1«]
Camera name [»CAMNAME«] = [» «]
Camera serial number [»CAMSERIAL«] = ####### [»#######«]
Drive dial position [»DRIVE«] = Single [»SINGLE«]
Exposure compensation (user-set) [»EXPCOMP«] = +0.00 [»+0.00«]
Exposure compensation (user plus bracketing) [»EXPNET«] = +0.00 [»+0.00«]
Exposure length [»EXPTIME«] = 12500 µs [»12500«]
Firmware version [»FIRMVERS«] = 2.00 [»2.00«]
Flash setting [»FLASH«] = Off [»OFF«]
Focal length [»FLENGTH«] = 16.6 mm [»16.6«]
Focal length (35mm) [»FLEQ35MM«] = 28 mm [»28«]
Focus status [»FOCUS«] = Auto focus locked [»AF«]
Imager board ID [»IMAGEBOARDID«] = [» «]
Temperature of sensor / header board [»IMAGERTEMP«] = 48 °C [»48«]
ISO speed [»ISO«] = 100 [»100«]
Lens aperture range [»LENSARANGE«] = 4.0 [»4.0«]
Lens focal length range [»LENSFRANGE«] = 16.6 mm [»16.6«]
Lens identifier [»LENSMODEL«] = [» «]
Shooting mode [»PMODE«] = Aperture priority [»A«]
Resolution of image [»RESOLUTION«] = Hi [»HI«]
Sensor type and revision [»SENSORID«] = [» «]
Shutter speed (exact) [»SHUTTER«] = 0.01250 s [»0.01250«]
Shutter speed (display) [»SH_DESC«] = 1/80 [»1/80«]
Date/Time original [»TIME«] = Aug 23, 2013, 2:00:45 PM [»1377266445«]
White balance setting [»WB_DESC«] = Auto [»Auto«]
–––––––––––––––
IMAGE DATA LIST
–––––––––––––––
1 : version : 2.0
: type : 2 - is processed for preview ? yes
: format : 18 - JPEG-compressed 8/8/8 RGB
size : 2640 x 1760 pixel
row size : 0 bytes
2 : version : 2.0
: type : 3 - is processed for preview ? no
: format : 30 - RESERVED
size : 2688 x 1792 pixel
row size : 13877536 bytes
3 : version : 2.0
: type : 2 - is processed for preview ? yes
: format : 18 - JPEG-compressed 8/8/8 RGB
size : 221 x 147 pixel
row size : 16846 bytes
| Module | Description |
|---|---|
| de.bernd_michaely.common.cli.parser |
This module contains utilities to provide a GNU getopt style
command line argument parsing.
|
| de.bernd_michaely.common.io.random |
This module contains classes for efficient random access of
miscellaneous data sources like files or byte arrays.
|
| de.bernd_michaely.x3f.extractor.cli |
This module provides a command line interface to the JX3fExtract library.
|
| de.bernd_michaely.x3f.types |
This module contains classes for creating instances of
parser objects for X3F (FOVb) files and types describing their content.
|
| de.bernd_michaely.x3i.types |
This module contains classes for creating instances of
parser objects for X3I (FOVi) files and types describing their content.
|