Home

User Manual - SMOS L1 Processor Prototype

1. i xs fname prepare_schema link_fname nr_links error i error create_data_file schema_filename objectID gt 1 xs size get_schema_size xsl_fname fname create dumy file size int load data file name xs fname error set char object ID vamame value B L T error set double object ID vamame value i 1 error save data file objectID bin filename gt i Figure 5 An example of how to use the Library to save information 2 5 Configuration File Before using the functions provided by the library it is necessary to read the configuration file using the function read_config_file The structure of the configuration file is as follows lt xml version 1 0 lt coniig lib home dir opt binxml fh lib home dir tmp data dir tmp data s tmp data dir dataset string dataset src dataset string header size string HEADER SIZE header size string top lines of header 25 top lines of header lt cag i block Mms eD Dm JE IUS HESS e eus Doek Matres tag block set DATA SET tag block set tag block set name DATA SET NAME tag block set name tag block set size MDR SIZE tag block set size tag block set count NUM MDR tag block set count tag block set offset MDR OFFSET tag block set offset tag byte order Byte Order c tag byte order gt config The tag lib home dir identifies the library path and is used by the function read
2. tError get double int object ID char varname double value tError get complex int object ID char varname tComplex value tError get doublecomplex int object ID char varname tDoubleComplex value tError get fixed array int object ID char varname tVector vector tError get variable array int object ID char varname tVector vector tError get vector byte int object ID char varname tVectorChar array char tError get vector long int object ID char varname tVectorLong v long EET norEdgqebmvesioreininmsobsjectm 1D chari varnam EVectorinti Qr oye iN tError get vector double int object ID char varname tVectorDouble array double tError get vector short int object ID chan varname tVectorShort array short tError get vector float int object ID char varname tVectorFloat array float This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Date 02 12 08 d ei m A S Critical EE XML Binary CFI Date ENGENHARIA J ye File Handling TER 47 of 17 tError get vector complex int object ID char varname tVectorComplex array complex tError get vector doublecomplex int object ID char varname tVectorDoublecomplex array dcomplex Ma tError get matrix int int object ID char varname tMatrixInt matrix int tError get matrix byte int object ID char varname tMatrixByte matrix byte
3. This function receives a short value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 6 set int Syntax RI norESccEnEnusuobyec o IDcharramnamcdmemvralmepgs This function receives an int value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR Syntax tError set long int object ID char varname long long value This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 7 Code d i m s Critical EE XML Binary CFI ind 02 ps H ENGEN ARIA J UE File Handling i 38 of 38 This function receives a long value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT
4. binx 2 Using the same dataset definition and creating a new type definition lt binx gt lt definitions gt lt defineType typeName Galaxy_Map_Type gt lt sCruct gt lt arrayFixed varName layer gt lt float 32 varName value gt lt dim indexTo 720 gt lt dim indexTo 1440 gt dim lt arrayFixed gt lt struct gt lt defineType gt Types defineType typeName Galaxy Map Record Wide definition SSE EIE arrayVariable varName record lt sizeRef gt unsignedByte 8 lt sizeRef gt lt useType typeName Galaxy_Map_Type gt lt dim gt lt arrayVariable gt lt struct gt lt defineType gt lt definitions gt dataset src byteOrder bigEndian gt lt useType typeName Galaxy_Map_Record varName Galaxy_Map gt Dataset lt dataset gt definition lt binx gt Another objective related with the proposed changes to the schemas would be avoiding the obligation of replicating the data element in the schema before reading data i e instead of calling the prepare schema function see 4 4 1 10 the function prepare schema nested would be called see 4 4 1 11 to insert only the proper data set source and byte order attributes With this change the diagram of Figure 3 would be simplified as shown in Figure 7 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 ares Dat
5. tError get char int object ID char varname char value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to char and is returned to the user in value The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 5 get str Syntax tError get str int object ID char varname char value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is returned to the user as an array of char null terminated a sting in value The function returns a tError structure to inform the status of the operation NOTE This function allocates memory for returning the string and the user must release it when it is not used anymore The errors returned are e ERR CODE ERROR ACCESSING OBJECT This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J alent File Handling e ah arab e ERR CODE NO ERROR 4 4 2 6 get short Syntax tError get short int object ID char varname unsigned short value This function requests the value of the variable identified by varname in the object of BinX ide
6. Double_vector 0 155555 Doublecomplex_vector R 0 155555 I 0 255555 Complex_vector R 0 3 I 0 4 Int_matrix 6454233 Float_matrix 33244 5 Long_matrix 28876544 Double_matrix 0 3343 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission N deim s EE XML Binary CFI ENGENHARIA J TEE File Handling Code SO UM DME L1PP 0005 Date 02 12 08 Issue 2 5 Page 49 of 49 Byte matrix Variable array of basic type 6 integers 2 4 8 16 32 64 Variable array of structure type 2 structures of 3 integer 2 4 8 and 32 64 128 The self test shall output the results on screen producing a report like the following one Reading Config File OK Getting the variable array of basic type value OE Getting the variable array of structure type value OK Creating object for write OK Objecte size irsk SSMS pos GRE Setting object types set short ec U0 Men OE sec Ant SOOO cos sS OS set floatr 350 543 00K set lon qe 00m NOR set double n 3510 0 9 0 iene ORI set complex r 0 2 i 0 6 OK set doublecomplex r 0 258 i 0 636 OK setting long vector 6745662 OK setting float vector 54637 8 OK set5ungsdoublesvectonus0 ms s5555 mr OK setting doublecomplex vector r 0 1555555 i 0 2555555 0K SAGES inte mateix Ul es css sete ing float matrix 339244 9 5 4 5 OR setting long matrix 288765
7. SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI icis ENGENHARIA J software File Handling bae pen The first time that it is run the configuration file does not exists so it is created and it is necessary to restart the test The test shall write known values into a Hybrid file using the write functions and it shall later open it and decode all the variables comparing them to the expected values An output like the one presented in Annex 4 shall be shown identifying the status of every function that can be used from the API If everything is OK and no error is shown the library is correctly installed and ready to use 2 9 Constraints As mentioned before the provision of the binary format in a schema allows altering such format directly into the schema without the need of recompilation of the tool using the CFI However the type of data of a particular variable contained in the binary file cannot change drastically from the original type so that the type used in the implementation still makes sense For example if a variable is written in the binary file as a char and afterward an integer is used to represent that same variable the library will not be able to return the correct value because surely the tool implementation using the CFI shall be expecting a char and the representation of a char is up to 255 This feature is present in this version 1 3 of the library which does not return any error if the user tries to
8. e ERR CODE NO ERROR 4 4 3 8 set float Syntax tError set float int object ID char varname float value This function receives a float value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 9 set double Syntax tError set double int object ID char varname double value This function receives a double value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 10 set complex Syntax tError set complex int object ID char varname tComplex value This function receives a complex value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR This document is property of DEIMOS Engenharia and cannot be distributed
9. var long unsignedByte 8 varName var char S SEXES defineType lt definitions gt lt dataset src binary_product byteOrder bigEndian gt useType typeName Basic Types varName B_Types gt dataset lt binx gt 2 8 Library Self Test To compile the test it is necessary to compile the BinXML library first and include the library directory on the LD_LIBRARY_PATH or DYLD_LIBRARY_PATH for MacOS users Then in the test directory run the make command to build the test program make test binxml The syntax of the program is the following test binxml To run the test it is simply required to invoke the name of the executable Then the test can be run in two different ways 1 call directly the tester passing as argument the configuration file bin test binxml binxml config xml 2 define the environment variable BINXML HOME as the home directory of the library and than run the test without specifying the configuration file In this case the library will be using the configuration file in its home directory as defined by the environment variable It is important to include the slash at the end of the path export BINXML HOME opt binxml fh bin test binxml test data hybrid binxml test eef schema templates doublecomplex template xml This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code
10. 12 02 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 77 Code SO UM DME L1PP 0005 o its EE XML Bi CFI Date 02 12 08 d 1 m s Critical bn Issue 2 5 ENGEN ARIA J TT File Handling qM Table of Contents iM U 1 PSI tese 1 1 2 Acronyms and Abbreviations ener oes vto oor ta epo e SP evo PY PP o IP Far Ero eee epe eee o rUR tee Eos eb Sa e ERs Pk e p Pbi epe E pei pU 1 1 3 Applicable and Reference Documenis 5 cierre betont rp np prae ae eo pP eo ee ope a eB soesscoocssscespensoeseecnndsestess 1 13 1 Applicable Documents iret bes eL EE Eo Ee IDEE S HEEL N 1 1 3 2 Reference Documents iin ine ERE RE PELA antenna AA i na ROSAE PARESE PANAR Esa REOR ARARR PE LA ARER 2 2 LIBRARY DUBIE iceszoteseebsesieriven unt EE TOENE 3 ZAM MOD COUN ccee 3 22 Compo neii E ciadenee E EE cus Soscuadesce cele becuchesmedscasubosuenasesucctondeve 3 ps Object Honde estes cr e UE 3 22 2 AP e E E IEEE 4 2 3 Compile and ce e M 5 DAU SABC E E A E E A E E N 6 pM Conticuration Filesi 9 2 6 Hybrid Me hi adet rerit eerie aseo cista eet aaae eo eode veu eek aae a 10 2 7 Template Tile ebrei qi etu me eise dvue os dee E E 11 2 8 Library Sell TOS me 11 PA NL cM P M 12 PAM SONdoaciuliagel rcr 12 JS BRIEF DESCRIPTION a
11. ASCII XML File Handling Library qe Binary XML File Handling Library 90x SceneASICs Figure 3 Retrieving a scene from a hybrid XML data file As it is shown in Figure 3 the CFI starts by accessing the header of the hybrid xml file to know how many scenes exists in the file and provides this information to the user who then requests the retrieval of a specific scene or an array of scenes To retrieve the required scene the user must implement a function that uses the API of the library to retrieve the information and fill the user s data structure For that purpose the user calls the function of the API extract data to extract the desired block of data scene from the hybrid xml file and write it into a binary data file Then the user will request the BinXML FH to prepare the schema that characterizes the block data scene that was extracted previously prepare schema With the binary file and the schema ready the user requests the BinXML FH to map into memory the binary file using the schema 1oad data file This returns an identifier that has to be used to retrieve the variables value in order to fill the user defined structure Load to structure This is done calling the proper get functions as required by the structure After the values of the variables have been inserted in the user defined structure the user can request the library to release the memory allocated to the binary file using the definition
12. CFI ind icis ENGENHARIA J software File Handling e ruis 4 4 1 16 copy file Syntax tError copy file char fname in char fname out This function copies the file name in to the file name out e ERR CODE ERROR WRITING FILE e ERR CODE ERROR READING FILE e ERR CODE NO ERROR 4 4 1 17 freeNestedData Syntax void freeNestedData void objectID This function destroys the Object that was used to read a Nested Dataset without releasing the memory structure allocated and returned with the execution of readNestedData 4 4 2 Retrieving Functions The following functions require that the 1o0ad data file function have already been called in order to obtain the object ID associated with object of BinX that contains the required variable An example of how can these functions be used to retrieve information is shown in Figure 4 4 4 2 1 get bit Syntax tError get bit int object ID char varname tStartBit StartBit int bit nr char value This function requests the value of the bit at position bit nr ofthe variable identified by varname in the object of BinX identified by object ID Then the value is converted to char 0 or 1 and is returned to the user in value The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 2 get bits Syntax cieco oer lowie os dclojecu JU
13. EEFrTOF get matrix float int object ID char varname tMatrixPloat matris float EErfror get matrix long int object ID char varname tMatrixLong matrix long tError get matrix double int object ID char varname tMatrixDouble matrix double tError get header size char hybrid fname int size tError extract header char hybrid fname char hdr fname int header size tError get data file info char filename char parent tDataBlockInfo info tError prepare schema char fname bin char fname link tIndices var indices char fname schema char endian tError prepare schema nested char fname bin char fname link char fname schema char endian tError extract_data char data filename char parent name tiIndices block_indices char bin filename tError load_data_file char schema_filename int objectID tError create data file char schema filename int obj size int objectID tError free data file int object ID tError save data file int objectID char bin filename terror set bie int Object ID chabrabmamermesSisab Bt St am Et Ine Witi nr Char valus tError set char int object ID char varname char value tError set str int object ID char varname char value tError set short int object ID char varname short value tError set int int object ID char varname int value tError set long int object ID char varname long long value tError set float int object ID char varname f
14. as in a list of consecutive similar data blocks the type is common to all of them but the value name shall change from one to the next one This has been treated in the same way as C arrays reserving the characters for it and the character for concatenation In this way a user wanting to access the NAME value in three consecutive SCENE data blocks would need to ask the CFI for the following values SCENE 1 NAME SCENE 2 NAME SCENE 3 NAME see 2 7 Template file 2 10 Degree of Portability The library has been tested on the following environments U SUSES8 RedHat Fedora This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Q nnns Date 02 12 08 deimos Li EE XML Binary CFI Pa E ENGENHARIA J software File Handling e Bog LJ SUN Solaris 5 8 MacOS X Darwin 7 9 0 More OS shall be added as the multi platform testing proceeds This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission de m5s TH EHOGENHARIA J software 3 BRIEF DESCRIPTION EE XML Binary CFI File Handling Code SO UM DME L1PP 0005 Date 02 12 08 Issue 2 5 Page 14 of 14 The following chapter provides a quick reference guide to definitions enumerated values structures error codes and function interfaces For a detailed guide the next chapter shall be use
15. at least one variable of each structure To solve this problem the CFI builds a BinX Schema file describing the structures the user wants and then uses the Object Handler to load all the structures required into memory When a structure is loaded a new entry is added to the table associating an identifier with that structure the Object Then the user requests to the API the structure s variable providing the identifier and the variable name The Object Handler then returns to the API the value of the required variable The Object Handler is also responsible for returning to the API the structure content of an object when requested i e given a structure the Object Handler must fill that structure retrieving the information from the object that represent the information loaded from the binary data file using BinX The library imposes no limits in the amount of data committed to memory although the user must be aware that loading more data than the available RAM shall most probably result in loss of performances 2 2 2 API The API is implemented in ISO C99 and represents the functions that are available for the user It shall return the values gathered from the Object Handler to the user This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind igi ENGENHARIA J alten File Handling Page 5 of 5
16. config file when the configuration file is passed to it as argument The tag tmp data dir indicates the directory where the temporary files needed by the library will be created These temporary files are the binary files and the schema files This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J saltare File Handling TER ano The tag dataset_string specifies a string that contains the tag and attribute of the template where it is indicated the variable type and name see 2 6 Hybrid file header The tag header_size_string specifies the tag from the header of the hybrid XML file that indicates the size of that same header The tag top lines of header indicates the number of lines from the header of the hybrid XML file that will be read looking for the string defined by the tag header size string Because the hybrid file may contain different types of data here referred as blocks the header of the hybrid file must contain a list with a specific number of blocks Each block must contain a name the offset to the data block starting from the beginning of the hybrid file the number of elements that are in the mentioned block and the size of each on of its elements The tags of the configuration file that define the tags of the block list contained in the header of the
17. directory otherwise the given file name will be read The function returns a tError structure to inform the status of the operation NOTE this function allocates memory that is necessary to release at the end of the program or when new configurations have to be loaded free config file The errors returned are ERR CODE ERROR FINDING ENV VAR ERR CODE TERMINATE e ERR CODE ERROR READING FILE e ERR CODE NO ERROR 4 4 1 3 free config file Syntax void free config file This function is used to release the memory allocated to the constant values read from the configuration file It is usually the last function to be called in the program and also when a new configuration file is to be read 4 4 1 4 join to hybrid file Syntax tError join to hybrid file char fname header char fname binary char fname hybrid This function is used to join a header and a binary data file into a hybrid XML file The header file name and the binary data file name should be the same as the hybrid file name only the extensions should be different eef for the hybrid file hdr for the header file and db1 for the binary data file The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ALLOCATING MEMORY e ERR CODE ERROR READING FILE e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 5 split hybrid file Syntax tError split hybrid _fil
18. ee SP ED attribute packed Typedef struct abge CXoEauE p t data array __attribute__ packed t odata t myarray lt xml version 1 0 encoding UTF 8 lt binx gt lt definitions gt lt defineType typeName Array_Type gt SISSE float 32 varName f integer 32 varName i double 64 varName d gt amp y GYESETDXGAE S defineType defineType typeName Data Type lt SCrucE gt lt arrayVariable varName record gt lt sizeRef gt lt unsignedInteger 32 gt lt sizeRef gt lt useType typeName Array_Type gt lt dim gt lt arrayVariable gt struct defineType definitions dataset src byteOrder bigEndian useType typeName Data Type varName My Array dataset sS binx The string describing the format of the memory structure is IP FIDP This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 N a e d el m O S Critical EE XML Binary CFI pate Oeria og Issue 2 5 ENGENHARIA vatiwars File Handling Page 58 of 58 What about nested variable arrays Example Considering the following user memory structure and the corresponding BinX schema typedef struct lt xml version 1 0 encoding UTF 8 gt char polID lt binx xmlns http www edikt org binx 2003 06 binx gt int polVal lt definitions gt attribu
19. elements contained in it U tVectorChar This structure is used to represent a one dimensional array of char with the additional information of the number of elements contained in that array Q tVectorLong This structure is used to represent a one dimensional array of long long C99 Standard with the additional information of the number of elements contained in that array U tVectorFloat This structure is used to represent a one dimensional array of float with the additional information of the number of elements contained in that array U tVectorDouble This structure is used to represent a one dimensional array of double with the additional information of the number of elements contained in that array U tVectorInt This structure is used to represent a one dimensional array of int with the additional information of the number of elements contained in that array U tVectorShort This structure is used to represent a one dimensional array of short with the additional information of the number of elements contained in that array Q tVectorComplex This structure is used to represent a one dimensional array of complex with the additional information of the number of elements contained in that array LC tVectorDoublecomplex This structure is used to represent a one dimensional array of doublecomplex with the additional information of the number of elements contained in that array L tMatrixInt This structure is used
20. function is used to retrieve an array of Long long C99 Standard and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tError structure to inform the status of the operation NOTE The usage of the C99 Standard is required because a long in 32 bit architecture is 32 bit and it is expected to have a representation of 64 bit as it happens with the Long long type provided by the C99 Standard The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 20 get vector float Syntax tError get vector float int object ID char varname tVectorFloat array float This function is used to retrieve an array of float and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 21 get vector double Syntax tError get vector double int object ID char varname tVectorDouble array double This function is used to retrieve an array of double and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tError structure to inform the status of
21. hybrid file are the following The tag block list defines the tag that identifies the beginning of the list of blocks This tag should contain an attribute count that specifies the number of blocks contained in the hybrid file The start of a block description is identified by the tag defined in tag block set The block name and offset are specified by the tags defined in tag block set name and tag block set offset respectively The offset of each data block is always related to the start of the file The number of elements contained in each block is specified by the tag defined in tag block set count and the size of each element is identified in the tag defined by tag block set size The tag tag byte order is used to identify the byte order used in the data block 2 6 Hybrid file header Here is a description of the required elements of the hybrid XML header The header of the hybrid file must contain at the beginning the tag specifying the size of the header This tag is defined in the configuration file by the tag header size string and must be in the first lines as defined by top 1ines of header in the configuration file The header must also contain a block list with the tags defined in the configuration file In the following paragraph there is an example of such a header lt FILE gt lt HEADER_SIZE gt 0000005367 lt HEADER_SIZE gt lt DSD_LIST count 01 lt DATA_SET gt lt DATA_SET_NAME gt CHARACTERS
22. inform the status of the operation NOTE 1 The schemas of BinX require that the binary file is in the same directory of the schema itself so the file name must be specified in the schema without the full path NOTE 2 This function allocates memory for returning the schema file name and the user must release it when it is not used anymore The errors returned are e ERR CODE ERROR READING FILE e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 12 load data file Syntax tError load data file char schema filename int objectID This function loads into memory the data contained in a binary file according with the XML schema contained in the file with name schema filename All the information can be accessed by the This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI icis ENGENHARIA J alien File Handling TE 26 of 26 variable name associated with the data It returns an integer objectID that identifies the object returned by BinX and that contains the information The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR READING FILE e ERR CODE NO ERROR 4 4 1 13 create data file Syntax tError create data file char schema filename int obj size int objectID Because BinX does not allow writing into a
23. load a variable of one type i e double with a different function i e get_int The library automatically assigns the proper values so the memory is not corrupted In the current version a warning is returned for any set function each time the user does not use the appropriate function for the type of variable being used Additionally get functions inform the user when the variable retrieved exceeds the size of the returning function i e get int used to retrieve double valued variable In order to decode a hybrid xml file the CFI needs to know how to separate the binary component from the XML header to apply separate schemas This is done by forcing the definition of the tag HEADER SIZE somewhere in the XML header containing the size in bytes of the XML header component This value shall be read by the CFI using default C reading functions as it shall not be possible to apply XML parse functions until the header is separated see 2 6 Hybrid file header This characteristic is needed to be able to read hybrid files whose binary component is not enclosed within tags XML parse functions are not able to decode a file whose complete content is not enclosed in tags and defining a binary part within an XML file could cause a parsing error although highly improbable as the ASCII representation of the binary characters could match a closing tag It is necessary as well to reserve some special characters for defining the BinX schema
24. lt DATA_SET_NAME gt lt MDR_SIZE gt 0000002 lt MDR_SIZE gt lt NUM_MDR gt 00075 lt NUM_MDR gt lt MDR_OFFSET gt 0005367 lt MDR_OFFSET gt lt Byte_Order gt 3210 lt Byte_Order gt lt DATA_SET gt lt DSD_LIST gt lt FILE gt This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Q rye Date 02 12 08 deim s Li EEXML Binary CFI 5 ENGENHARIA J software File Handling e ara 2 7 Template file The template file is used to generate the schemas required by BinX to read the binary file It is a XML schema file following the specification of BinX BinX but with some restrictions Q in the schema template after the tag defined by dataset string in the configuration file only the first line containing the useType tag will be taken into account when creating the schema using the prepare_schema function Any tag and or comment after this line is ignored L the variable name must contain the character that identifies the place where the index of the element will be placed Following it is an example of a template lt xml version 1 0 encoding UTF 8 lt binx xmlns http www edikt org binx 2003 06 binx gt lt definitions gt lt defineType typeName Basic_Types gt SUC unsignedInteger 32 varName var integer float 32 varName var float double 64 varName var double gt unsignedLong 64 varName
25. of the schema using the identifier that was returned previously ree data file After this process the user has stored the requested scene in its own defined structure using the memory allocated by him In the following image the previous process is shown also providing a user defined function get scene which includes all the CFI function calls This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 7 Code SO UM DME L1PP 0005 itini Date 02 12 08 deim s lu rb a is ENGENHARIA Ur software File Handling Page 8of8 Logical library library Model L1PP Logical Logical Model BIN_XML FH Model ASCII XML FH tError get data file info header fname parent info fid xf tree init parser data file error u i xf tree get fixed header item fid Nr of scenes value error Li error get scene data file name nr of sceneslist of scenesthe scene bin fname extract data data fname data indices data size error fid xf tree init parser file error T xf_tree_get_fixed_header_item fid data_offset value error xs fname prepare_schema link_fname nr_links error int load data file bin fname xsl fname load_scene object_ID theScene error char get_char object_ID var_name L t L i L i i i get_struct object_ID var_name struct
26. or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind iis ENGENHARIA J alent File Handling e 39 of 39 4 4 3 11 set doublecomplex Syntax tError set doublecomplex int object ID char varname tDoubleComplex value This function receives a double complex value to be put on the variable varname of the BinX object identified by ob ject ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 12 set vector byte Syntax tError set vector byte int object ID char varname tVectorChar v char This function receives an identification of an object already loaded object ID and set the variable varname with the content of the char vector v char The type of this vector is tVectorChar and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 13 set vector short Syntax tEcconlset vector Ine Ane mobyeccm ID cham Avanname st Vectorohort iv Short This function receives an identification of an object already loaded object_ID and set the variable varname with the content of the short vector v short The type of this vector is tVectorShort and his definition is described in the sec
27. returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 17 get vector short Syntax tError get vector short int object ID char varname tVectorShort array short This function is used to retrieve an array of short and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 18 get vector int Syntax tError get vector int int object ID char varname tVectorInt array Ine This function is used to retrieve an array of int and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tError structure to inform the status of the operation The errors returned are This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J alent File Handling e quias e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 19 get vector long Syntax tError get vector long int object ID char varname tVectorLong v long This
28. to represent a two dimensional array of int with the additional information of the number of elements contained in that array QO tMatrixByte This structure is used to represent a two dimensional array of byte values with the additional information of the number of elements contained in that array tMatrixFloat This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI iis ENGENHARIA J software File Handling TE iratos This structure is used to represent a two dimensional array of float values with the additional information of the number of elements contained in that array tMatrixLong This structure is used to represent a two dimensional array of 1ong with the additional information of the number of elements contained in that array L tMatrixDouble This structure is used to represent a two dimensional array of double with the additional information of the number of elements contained in that array tMatrixComplex This structure is used to represent a two dimensional array of complex with the additional information of the number of elements contained in that array LQ tMatrixDoublecomplex This structure is used to represent a two dimensional array of doublecomplex with the additional information of the number of elements contained in that array Q tIndices This is the same
29. void objectID This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J software File Handling TE Ror 48 4 DETAILED DESCRIPTION 4 1 Defines Q CONFIG FILE It defines the name of the configuration file of the library MAX FILE NAME SIZE It defines the maximum size allowed for identifying a file with the full path MAX ERROR MSG SIZE It defines the maximum size of the message describing an error as used in t Error MAX LINE SIZE It defines the maximum size of the string read from an ASCII file It is used when preparing a schema with the functions prepare schema and prepare schema nested MAX XPATH SIZE It defines the maximum size of a string containing the XPath used to read write values from an XML file using the EEFH RD 4 4 2 Enumerated Values 4 2 1 eErrorCode Error Codes It identifies the type of error that has occurred C NO ERROR Error used to inform that no error have occurred during function processing U ERROR ACCESSING OBJECT This error occurs when the provided ob ject ID doesn t exist ERROR ALLOCATING MEMORY This error is used to inform that there was a problem allocating memory C ERROR FINDING ENV VAR This code is used when the required environment variable couldn t be found probably because it is
30. 0 or 1 StartBit defines the start of counting for bit nr as described previously in section 4 2 3 The function returns one of the enumerated tError values according to the status of the operation 4 4 3 3 set_char Syntax tError set_char int object_ID char varname char value This function receives a char value to be put on the variable varname of the BinX object identified by object ID The value is converted to the format defined in the schema It returns one of the enumerated tError values according to the status of the operation This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J alien File Handling e The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 4 set str Syntax tError set str int object ID char varname char value This function receives a null terminated string value to be put on the variable varname of the BinX object identified by ob ject ID The value is converted to the format defined in the schema It returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 5 set short Syntax tError set short int object ID char varname short value
31. 2 3 Compile and link The library has the following dependencies Compilation Library Version Library Name libxml2 2 6 22 or XML C parser and toolkit newer libexplorer_file_handling Ac Earth Explorer File Handling libxerces c 2 7 0 or Apache Xerces XML parser newer Table 4 Library dependencies Since the BinX library was discontinued the dependency from the BinX was removed Instead BinXML FH now includes a modified version of BinX library BinX 1 2 4 which is not an official release and is only used internally by BinXml FH The libxml2 library is required by the CFI library The Xerces library is required by BinX For more details on how to install each one of the libraries please refer to each library s documentation The dependencies of the current library are only on the EE File Handling CFI and on the BinX library These previous libraries and not the library described in this document impose the dependency on libxml and libxerces Tests have been made with more advanced versions e g libxml 2 6 31 without any problem the shown recommended versions are taken from the BinX and EE CFI documentation The library is structured as shown in Figure 2 where the src directory contains the source files c cc the include directory contains the headers h required to compile the library s components and the obj directory contains the objects resulting from the compilation of the sources The library will be cr
32. 44 OK setting double matrix 0 3343 OK Testing save data file OK This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Date 02 12 08 Issue 2 5 Page 50 of 50 d ei mos Critical EE XML Binary CFI ENGENHARIA J yeu File Handling ck ck ck ck ck ck ck ckckckckckckckckckckckckckck Reading Written Data KKEKKKKKKKKKKKKKKKKKKKK Extracting Header OK Getting the Short Value 300 OK Getting the Int Value 3000 OK Getting the Float Value 350 543 OK Getting the Long Value 3500768 OK Getting the Double Value 3 500100e 03 OK Getting the String Value test binxml fh OK Getting the Complex Value r 0 200000 i 0 600000 OK Getting the DoubleComplex Value r 0 258000 i 0 636000 OK Getting the matrix double Value OK This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 7 Code SO UM DME L1PP 0005 e d eim s EE XML Binary CFI Pte 02 12703 Issue 2 5 ENGENHARIA J mre File Handling Page 51 of 51 Getting the matrix_byte Value OK Getting the Bit 5 of Short Value 1 OK cecting che Bies Oito S RES Mc eT TA RTTNQ 0 MO OK 8 2 test nested The library functionality with nested complex data types e g arrays fixed or variable that include other arrays is tested with test nested while test bi
33. Byte matrix byte This function is used to retrieve a two dimension array of byte values and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 25 get matrix float Syntax theconlget matrix Elbat GOnte ebJect IDI chari ivarname EMatrikiloat matrik Eilcan p This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI nd icis ENGENHARIA J software File Handling TE de ai an This function is used to retrieve a two dimension array of float and the number of elements in that array contained in the variable with name varname associated with the object identified by ob ject ID The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 26 get Syntax tErrOrF get matrix long int object ID char varname tMatrisLhong matris long 5 This function is used to retrieve a two dimension array of 1ong and the number of elements in that array contained in the variable with name varname associated with the object identified by object I
34. D The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 27 get matrix double Syntax tError get matrix double int object ID char varname tMatrixDouble matrix double This function is used to retrieve a two dimension array of double and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 28 readNestedData Syntax void readNestedData char xml fname void objectID This function is used to read the BinX file and create a memory structure that contains all the information retrieved from each element of the BinX file particularly for files that are represented as linked list in memory It receives an xnl fname that corresponds to the BinX file name and an objectID where it is stored the reference to the Object that was used to read the BinX schema This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Q md Date 02 12 08 deim gt s lisi EE XML Binary CFI Pe E ENGENHARIA J alent File Handling TE 36 of 36 During its execution this function all
35. D Chee veuzes ame Jolt State aime onle Gwo uc GO Coers SESSEEUUUIS p This function requests the value of the bits at positions between bit start and bit stop ofthe variable identified by varname in the object of BinX identified by object ID Then the value is converted to a string of 0 and 1 and is returned to the user in value The function returns a tEr ror structure to inform the status of the operation This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI nd icis ENGENHARIA J ME File Handling TE 28 of 28 NOTE This function allocates memory for returning the string version of the bits and the user must release it when it is not used anymore The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 3 get byte Syntax tError get byte int object ID char varname char value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to char and is returned to the user in value It is expected that the variable contains a number and that number is lower than 256 The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 4 get char Syntax
36. EN a7 tComplex typedef struct doublecomplex double r double i tDoubleComplex typedef struct _Vector void value unsigned long count tVector typedef struct _VectorChar char value unsigned long count tVectorChar typedef struct VectorInt int value unsigned long count tVectorInt typedef struct _VectorLong long long value unsigned long count tVectorLong typedef struct _VectorFloat float value unsigned long count tVectorFloat typedef struct _VectorDouble double value unsigned long count tVectorDouble typedef struct _VectorShort short value unsigned long count tVectorShort typedef struct _VectorComplex tComplex value unsigned long count jtVectorComplex typedef struct _VectorDoublecomplex tDoubleComplex value unsigned long count tVectorDoublecomplex typedef struct MatrixInt ines value Code Date Issue Page SO UM DME L1PP 0005 02 12 08 2 9 150of 15 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 77 Code SO UM DME L1PP 0005 or Date 02 12 08 deimos Li EE XML Binary CFI Pa E ENGENHARIA J software File Handling Page 16 of 16 unsigned long rcount unsigned long ccount jtMatrixInt typedef struct MatrixByte chars value unsigned long rcount unsigned long ccount tMatrixByte typedef struct MatrixFloat
37. N 1 1 Purpose and Scope This document describes the EE XML Binary CFI File Handling Library for accessing read write binary files This CFI is to be used initially in the development of the SMOS L1 Processor Prototype in order to read already created binary files and write the different level products of the L1 Processor Detailed descriptions of each of the functions provided by the CFI are described here for reference The library is developed in such a way that it is independent of the SMOS mission being possible to use it for any kind of XML Binary files that follow some specific guidelines The functions presented in this document shall not make any explicit reference or assumptions related exclusively to the SMOS L1 Prototype 1 2 Acronyms and Abbreviations API Application Programming Interface CHI Customer Furnished Item Hybrid XML A file with a header written in XML concatenated with the binary information EEFH Earth Explorer File Handling CFI ASCII XML library LSB Least Significant Bit MSB Most Significant Bit EE Earth Explorer Table 1 Table of Acronyms For the complete list of acronyms please refer to the document SO LI CASA PLM 0094 Directory of Acronyms and abbreviations 1 3 Applicable and Reference Documents 1 3 1 Applicable Documents SOW SO SOW CASA PLM Level 1 Processor Prototype Development Phase 3 01 0855 and Support and Analysis Activities Statement of Work BinX BinX Editkt BinX 1 2 D
38. OBJECT e ERR CODE NO ERROR 4 4 2 14 get fixed array Syntax tError get fixed array int object ID char varname tVector var vector This function fills a structure of type t Vector whose element count states the number of elements in the array pointed by the pointer in t Vector element value The memory pointed by value contains the values read from the file according to the schema It is the responsibility of the caller to cast the void in value to the correct structure and any change in the schema related with the fixed array must be taken into account on the caller s structure The function returns a tEr ror structure to inform the status of the operation SPECIAL NOTE As opposed to the previous access methods the structure accessed through this one may not change in runtime i e altering the BinX schema as the structure NEEDS to be known before hand to the function invoking the CFI Any change in the structure format will ALWAYS require recompilation to avoid errors not in the CFI but in the functions using it The errors returned are ERR CODE ERROR ACCESSING OBJECT ERR CODE NO ERROR ERR CODE INVALID DATA TYPE ERR CODE ERROR ALLOCATING MEMORY 4 4 2 15 get variable array Syntax tError get variable array int object ID char varname tVector var vector This function fills a structure of type t Vector whose element count states the number of elements in the array pointed by the pointer in t Vector element
39. S Word 2000 File Name SO UM DME L1PP 0005 BINXML FH SUM Archive Code P SUM DME 03 013 036 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission d ei m O S EE ANKAN CFI ENGENHARIA Ue software File Handling Document Status Log Code Date Issue Page SO UM DME L1PP 0005 02 12 08 2 5 iii Change description Approved Table of Contents Delivery version for ADR Delivery after reviews from ADR Delivery with the release of library F A T vO Release of library version 1 3 Updates after CDR RIDs Updates for version 1 4 of library including variable arrays Updated for version 3 0 of the library Data Set offset starts from the beginning of the binary part of the product Updated for version 3 2 of the library BinX dependency was removed and dependencies updated Description of Nested Variable Arrays in Future Work section Description of Nested Variable Arrays Updated for version 3 5 of the library Modifications in the support of nested variable arrays Updated for version 3 6 of the library Updated descriptions of prepare schema nested Updated for version 3 7 of the library Added information about new function get fixed array and about test nested 2004 10 30 2004 12 20 2005 02 21 2005 02 23 2005 03 30 2005 05 02 2005 08 31 2005 09 23 2006 07 20 2006 11 17 2007 03 28 2007 11 28 2008 04 18 2008 07 22 2008
40. TellOeie s sedie unsigned long rcount unsigned long ccount tMatrixEloat typedef struct MatrixLong long longs irae unsigned long recount unsigned long ccount tMatrixLong typedef struct _MatrixDouble double value unsigned long recount unsigned long ccount tMatrixDouble typedef struct _MatrixDoublecomplex tDoubleComplex value unsigned long rcount unsigned long ccount tMatrixDoublecomplex 3 4 Function List tError append file char fname in char fname out tError copy file char fname in char fname out tError create config file char ConfigFileLocation tError read config file chbar config file name void free config file tError join to hybrid file char fname header char fname binary char fname hybrid tError split hybrid file char fname hybrid emai oet lode lite ObJece IUD char Casus eS Sessel Wine brenr Char valua GEC ror oet bits One Goyer JUD ekar wewcocW sy ote oni Start ime ajo loa Sieiers gher Commence ehar value tError get charline object ID char varname char value tError get byte int object ID char varname char value tError get str int object_ID char varname char value tError get short int object ID char varname short value tError get int int obyect rbD char varname int value tError get long int object ID char varname long long value tError get float int object ID char varname float value
41. This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI dee ipis ENGENHARIA J software File Handling Page 46 of 46 L BxDataset getArray int index Get a reference to the BxArray instance in the current position of the ordered list of dataset members QO BxDataset getDataset Get a reference to the BxArray instance in the current position of the ordered list of dataset members L1 BxDataset getDataObject char Name BxDataset getDataObject int index Get a reference to an object in a dataset based on the user defined variable name of that object or its relative position in the dataset BxDataset getUnsignedShort16 char varName LC BxDataset getUnsignedShort16 int_ index Get a reference to a BxUnsignedShort16 object contained in a dataset based on its variable name or relative position BxDataset getUnsignedLong64 char varName QO BxDataset getUnsignedLong64 int index Get a reference to a BxUnsignedLong64 object contained in a dataset based on its variable name or relative position LC BxDataset addDataObject Append a specified instance of BxDataObject like BxArray BxUnsignedB yte8 to a dataset BxDataset readFromFile Read the application data values for all dataset member objects from the associated binary file LC BxDataset toStreamBinar
42. _var H a Hi i free data file object ID error Figure 4 Example of how to use the Library to retrieve information The Figure 4 shows in more detail the sequence of the functions of the BinXML FH that are called to implement the user function that retrieves the data block Scene and fills the user defined structure as described previously In this figure the EEFH is identified as ASCII XML FH To perform the reverse process store the data block it is necessary to prepare the schema or use an existing one if that is the case that describes the way the information is to be stored prepare schema and then create a representation of the binary file in memory using that schema 1oad data file This returns an identifier that is used to pass the information to the API using the set functions After all the information have been passed the file can be written to disk save data file An example of the saving process is shown in Figure 5 where a user defined function save scene is also represented grouping the CFI use in a single function This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 EE XML Binary CFI Pete ae Issue 2 5 ENGENHARIA Ur software File Handling Page 9 of 9 Logical library Model L1PP Logical Model BIN XML FH I I 1 save_scene
43. as tVectorInt only it used to be more meaningful 4 4 Function Description 4 4 1 Initializing and Terminating functions 4 4 1 1 create config file Syntax tError create config file char ConfigFileLocation This function creates a default configuration file There are two possibilities for the location of that file 1 if the ConfigFileLocation is passed as NULL it will be required that the environment variable BINXML HOME is set to the home path of this library with the ending slash for example BINXML HOME opt binxml In this case the configuration file will be created in the library home directory 2 if the ConfigFileLocation is not NULL the file will be created in the specified directory The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 2 read config file Syntax tError read config iiile char config file name This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 a Code d 1 mos Critical EE XML Binary CFI nd 02 n H ENGEN ARIA J em File Handling oe ee This function reads the library s configuration file It follows the same approach as the function create config file ifthe config file name is NULL the configuration file will be read from the library s home
44. binary file according to an XML schema this function loads a dummy binary file of size obj size using the schemas stored in schema filename in order to obtain the identifier ob ject ID of an object of BinX that respects the data structure defined in the schema This object can be filled with the required values and then be stored to disk using save data file The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE NO ERROR 4 4 1 14 free data file Syntax tError free data file int object ID This function releases the memory associated with the object identified by object ID and that was previously allocated with 1oad data file or with create data file The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR REMOVING OBJECT e ERR CODE NO ERROR 4 4 1 15 append file Syntax tError append file char fname in char fname out This function appends the file fname_in to fname out If it is called several times it will always put the content of name in to the end of the file name out e ERR CODE ERROR WRITING FILE e ERR CODE ERROR READING FILE e ERR CODE NO ERROR This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary
45. c number of variables of the same type as defined by the var_indices see 2 6 Hybrid file header The new schema is updated specifying the byte order that is used in the binary data big endian or little endian and is stored in a file whose name is returned in fname_schema The function returns a tError structure to inform the status of the operation NOTE 1 The schemas of BinX require that the binary file is in the same directory of the schema itself so the file name will be specified in the schema without the full path NOTE 2 This function allocates memory for returning the schema file name that the user must release when it is not used anymore The errors returned are e ERR_CODE_ERROR_READING_FILE e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 11 prepare schema nested Syntax tError prepare schema nested char fname bin char fname template char fname schema char endian This function uses a schema template with file name name template describing a nested structure of data to create another schema that is stored in a file whose name is returned in name schema The byte order of the stored binary data big endian or little endian can be set passing endian b or 1 or can be specified in the schema template In the later case the user should pass endian 0O and the byte order of the template is returned in the same parameter The function returns a tEr ror structure to
46. char varname tMatrixLong matrix long This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 ram S Code d i m s Critical EE XML Binary CFI ind 02 bs H ENGEN ARIA J software File Handling Page 42 of 42 This function receives an identification of an object already loaded object ID and set the variable varname with the content of the long two dimension array matrix long The type of this vector is tMatrixLong and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 23 set matrix double Syntax tError set matrix double int object ID char varname tMatrixDouble matrix double This function receives an identification of an object already loaded object ID and set the variable varname with the content of the double two dimension array matrix double The type of this vector is Mat rixDouble and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 24 set matrix byte Syntax tError set matrix byte int object ID char varname tMatrixByte matrix byte This function receives an identification of an object already loaded object ID and set the variable varname with the content of the byte value two dimension array matrix byte The type of this vector i
47. d 3 1 Defines define CONFIG FILE binxml fh conf xml define MAX FILE NAME SIZE 256 define MAX ERROR MSG SIZE 256 define MAX LINE SIZE 256 Max size of a line read from an ASCII file define MAX_XPATH_SIZE 512 used to define the Max size of a XPath used in XML 3 2 Enumerated Values enum ERR CODE NO ERROR ERR CODE ERROR ACCESSING OBJECT ERR CODE ERROR ALLOCATING MEMORY ERR CODE ERROR FINDING ENV VAR ERR CODE ERROR FINDING STRING ERR CODE ERROR READING FILE ERR CODE ERROR REMOVING OBJECT ERR CODE TERMINATE ERR CODE ERROR WRITING FILE ERR CODE XML CFI ERR CODE INVALID OFFSET ERR CODE INVALID DATA TYPE eErrorCode enum ERR LEV LOW ERR LEV MEDIUM ERR LEV HIGH eErrorLevel enum MSB LSB eStartBit 3 3 Structures Types typedef eStartBit tStartBit typedef eErrorCode tErrorCode typedef eErrorLevel tErrorLevel typedef struct _Error tErrorCode ErrorCode tErrorLevel ErrorLevel char ErrorMsg MAX ERROR MSG SIZE CELECO typedef struct DataBlockInfo This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission d ei mos Critical EE XML Binary CFI ENGENHARIA J Tee File Handling char name long offset long element size long count char endian tDataBlockInfo typedef struct Indices mE Counte int value tIndices typedef struct _complex float ep ELG
48. data structure of nested7 has variable arrays at all the three levels The test should output to the screen a report like the following Data File unused dbl Schema File test data nested5 b xml Size of dataset 61 Size of buffer with pointers 9 Testing test data nestedb5 b xml Data File unused dbl Schema File test data nested5 l xml Size of dataset 61 Size of buffer with pointers 9 Testing test data nested5 l xml Data File unused dbl Schema File test data nested6 b xml Size of dataset 61 Size of buffer with pointers 27 Testing test data nested6 b xml Data File unused dbl Schema File test data nested6 l xml Size of dataset 61 Size of buffer with pointers 27 format 4BP B 4SP 2 BI P SP OK BP B SP 2 BI P SP OK format format B2 B SP 2 BI P S OK This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Date 02 12 08 d ei m A S Critical EE XML Binary CFI Date is ENGENHARIA J software File Handling T sro Testing test data nested6 l xml forma 8 WEVA Ie ASIP 2 EME 12 St OK Data File unused dbl Schema File test data nested7 xml Size of dataset 426 Size of buffer with pointers 12 Testing test data nested7 xml format IP SIII SP FFFFF SP SP P P OK 8 3 file_transform Another program provided is bin file_transform and the purpose of this is to split hybrid fi
49. defineType typeName Map_Type gt lt struct gt lt arrayFixed varName layer gt float 32 varName value gt lt dim indexTo 720 gt Types lt dim indexTo 1440 gt definition lt dim gt lt arrayFixed gt lt struct gt lt defineType gt lt definitions gt sdaasesEsnc uo vitse Oz oreo acim clen tii DOSES useType typeName Map Type varName Galaxy_Map gt Bc A dataset templateSize unit bytes gt 4155844 lt templateSize gt SUID EIER This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Zg 27728 E Date 02 12 08 d Critical EE XML Binary CFI TEn S File Handling S 54 bs The previous schema could be adapted to follow the format using variable arrays by 1 Using the same type definition and changing the dataset definition lt binx gt lt definitions gt lt defineType typeName Galaxy_Map_Type gt lt sstruct gt lt arrayFixed varName layer gt lt float 32 varName value gt lt dim indexTo 720 gt Types lt dim indexTo 1440 gt definition lt dim gt lt arrayFixed gt IS C EUCES lt defineType gt lt definitions gt dataset src byteOrder bigEndian gt arrayVariable varName Galaxy Map lt sizeRef gt lt unsignedByte 8 gt lt sizeRef gt Datasets lt useType typeName Galaxy_Map_Type gt Gieicxad ies om dim arrayVariable dataset
50. deim os liii ENGEN uU d software EE XML Binary CFI File Handling User Manual Code SO UM DME L1PP 0005 Issue 25 Date 02 12 08 Name Function Signature Prepared by N Almeida Project Engineer M Ventura Project Engineer A Lopes Project Engineer Checked by J Freitas Quality A Manager Approved by J Barbosa Project Manager DEIMOS Engenharia Av D Joao Il Lote 1 17 Torre Zen 10 1998 023 Lisboa PORTUGAL Tel 351 21 893 3017 Fax 351 21 896 9099 E mail mailto deimos deimos com pt DEIMOS Engenharia 2007 All Rights Reserved No part of this document may be reproduced stored in a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording or otherwise without the prior written permission of DEIMOS Engenharia Code SO UM DME L1PP 0005 o a aa nt EE XML Bi CFI Date 02 12 08 d IMs Critical bini Issue 2 5 NGEN ARIA Jr software File Handling Page 1 This page intentionally left blank This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 3 Q 4 Code SO UM DME L1PP 0005 HARIA J aren File Handling ME 2 5 Page ji Document Information Internal Public Contract Number DE04 B 434 P Industry Contract Issuer EADS CASA Espacio Confidential Internal Distribution External Distribution Josep Closa EADS CASA Espacio Michele Zundo Archiving Word Processor M
51. e 02 12 08 deim gt s liii EE XML Binary CFI Pe E ENGENHARIA 7 xeltsan File Handling aae err Hybrid File User User Memory Structure 9 j ereeer Read Access User Core functions ASCII XML File Handling Library Binary X ML File Handling Library Figure 7 Retrieving data from a hybrid XML data file using the variable array approach The procedure of reading data is almost the same with a minor change that is the introduction of a counter number of elements in the variable array at the beginning of the extracted binary file as shown in Figure 8 Hybrid XML file Binary Data File Binary Data Block Figure 8 Generating the binary file using variable array approach This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 4 Code SO UM DME L1PP 0005 Q VEL Date 02 12 08 deim s Li EE XML Binary CFI Pa i ENGENHARIA J software File Handling Page 56 of 56 9 2 Limitations There have been identified a couple of limitations related to the usage of nested variable arrays 1 Format 1 1 Variation of structures changes in the order by which the elements of the structure are defined in memory must be followed with the corresponding change in the order by which the elements are defined in the schema and vice versa The same is applicable to any change in the type of data either in me
52. e char fname_ hybrid This function is used to split a hybrid XML file into a header file and a binary data file This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI icis ENGENHARIA J m File Handling TE TR The new files are created on the same place that the original one and will have the same name only the extension will change from eef to hdr for the header file and db1 for the binary data The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ALLOCATING MEMORY e ERR CODE ERROR FINDING STRING e ERR CODE ERROR READING FILE e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 6 get header size Syntax tError get header size char hybrid fname int size This function returns in the argument size the size of the header contained in the hybrid file The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ALLOCATING MEMORY e ERR CODE ERROR FINDING STRING e ERR CODE ERROR READING FILE e ERR CODE NO ERROR 4 4 1 7 extract header Syntax tError extract header char hybrid fname char hdr fname int header size This function copies the header of the hybrid XML file of name hybrid fname and stores it on another file whose name is returned in
53. e enclosed within tags The CFI is based on BinX http www edikt org binx download index jsp which provides the storage and retrieval of data into binary files and is also based on the Earth Explorer File Handling library for storage and retrieval of data into XML files The CFI shall provide all the mechanisms internally to separate the XML and the binary parts for separate access making it transparent to the user Each part will need an independent validation schema describing the format and type of the values to be read from the XML and binary contents The BinX based schema and functions provide a direct conversion from the binary part contents into memory variables whose type is defined in the schema The intended use of this library is for the decoding of hybrid binary XML product files within the SMOS L1 Processor Prototype This library shall be the only interface for handling read and write functions within the prototype Handling of pure XML product files shall be covered by the already available EE File Handling CFI 2 2 Components The library is separated in two different components the component that deals with objects and that is related with the fact that the BinX library is written in C i e the Object Handler and the component that provides the functions that are available for the user in ANSI C i e the API 2 2 1 Object Handler The Object Handler is implemented in C and linked with the API The main goa
54. eRef lt byte 8 byteOrder littleEndian gt lt sizeRef gt lt useType typeName dstype varName dsname byteOrder littleEndian gt lt dim gt lt dim gt lt arrayVariable gt lt dataset gt lt binx gt The format describing this memory structure is BP B SP 2 BIDP SP Does it works for writing data The memory format is of utmost importance for writing data since that is the only way to access the information namely reading the counters of variable arrays This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission
55. eated in the 1 ib directory The test directory contains a program that can be used to check if the library is properly installed There are some templates in the directory schema templates and one hybrid file in test data that are used by this test program For more information on how to run it please see Library Self Test This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d im s Critical EE XML Binary CFI ind os H ENGENHARIA J software File Handling Been 6 of 6 TRIS a lai include api common L object handler Lio obj api common L object handler Schema templates SEE api common L object handler Figure 2 Directory structure of the library To compile the library it is necessary to use the Makefile that exists on the root of the library There is another dedicated Makefile in the test directory which shall be used to generate the self validation test This validation test can also be used as an example of how the API functions shall be used For the latest information on how to install the library please check the file xeadme txt located at the root directory 2 4 Usage The library has its own configuration file binxml fh conf xml that allows configuring some required parameter of the library see 2 5 Configuration File It is necessary to give permission to the user of
56. ent of the float vector v_float The type of this vector is tVectorFloat and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 17 set vector double Syntax tError set vector double int object ID char varname tVectorDouble v double This function receives an identification of an object already loaded object ID and set the variable varname with the content of the double vector v double The type of this vector is tVectorDouble and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 18 set vector complex Syntax tError set vector complex int object ID char varname tVectorComplex v complex This function receives an identification of an object already loaded object ID and set the variable varname with the content of the complex vector v complex The type of this vector is tVectorComplex and his definition is described in the section 3 3 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA JF software File Handling Page 41 of 41 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 19 set vector doublecomplex Syntax tError set vector doublecomp
57. eveloper s Guide 1 2 Table 2 Applicable Documents This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 deimosI E cmo i P ENGENHARIA qr are File Handling T are 1 3 2 Reference Documents CS MA DMS GS 0001 Earth Explorer Mission CFI Software MISSION CONVENTIONS DOCUMENT PE TN ESA GS 0001 Earth Explorer Ground Segment File Format Standard CS MA DMS GS 0002 Earth Explorer Mission CFI Software GENERAL SOFTWARE USER MANUAL CS MA DMS GS 0008 EXPLORER FILE HANDLING Reference Manual SO LI CASA PLM 0094 Directory of Acronyms and abbreviations EM Table 3 Reference Documents This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J software File Handling basa 3 of 3 2 LIBRARY GUIDE 2 1 Objectives The main objective of this CFI is to provide access to the data of a binary file using a XML Schema that defines the order and type of the data contained in that binary file This approach makes it possible to alter the read write format of a given binary file by simply changing the XML Schema The CFI shall also provide access capabilities to hybrid XML Binary files where the binary file has a XML ASCII header preceding it The binary content in such files shall not b
58. hdr_fname The function returns a tEr ror structure to inform the status of the operation NOTE This function allocates memory for returning the header file name and the user must release it when it is not used anymore The errors returned are ERR CODE ERROR ALLOCATING MEMORY ERR CODE ERROR FINDING STRING e ERR CODE ERROR READING FILE ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J software File Handling page 24 of 24 4 4 1 8 get_data_file_info Syntax tError get data file info char filename char parent tDataBlockInfo info This function retrieves the description information of a data block described in the filename hybrid s file XML ASCII header under tag DATA and whose name tag is equal to the string parent passed as argument The tDataBlockInfo structure with the information of the data block is returned to the user in value The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR FINDING STRING e ERR CODE NO ERROR 4 4 1 9 extract data Syntax tError extract_data char data_filename char parent name tIndices block indices char bin filename This function extracts one or several data eleme
59. i nnkeEERERERERRUXRER KRERER XX EXNRRARERRER FIXE FERE GRAB EXE saso aE 14 SNC 14 3 2 Enumerated Values eese o nib rsa Eea onum sons Er E O aaa RO ES ESET 14 Ee I EIA ATAN RIT E LC E LE 14 gm jani d 16 4 DETAILED DESCRIP TION ueicsiaenkbxkonp uh eno ancondebnconindu dodo icon aaron inpr on anoo d ten 18 J T DeFInGS noie DEM EGRE UIS EQUES IPA VL VASTAVET sotcateeessdavessondavabenddavessssdaveseaseaubensedusvbasadavsseonqasbbonedanbesesds 18 4 2 Enumerated Vales T PIR SENNSS 18 takeena ode Error C OTE ea EE E TA 18 42 2 eEmorLbevel Bsror Level iso a R E ai ia 19 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 deim s Critical EE XML Binary CFI Date A ENGENHARIA J T File Handling Page V 22 3 estan Bi Start Bt n tech eite met td etae DU e a a edi Vea ERA C cv REED y 19 4 3 Structures Types m 19 4A Function DOS CHIC OI iiss sce2scctessssexsaclessocasvesvosuncoeeesdesessessadeseacuvsasvecuacessaobsaessesbueessqcbebetsecuncetsecdcvonsexcssers 21 Ad l tial zane and berminatumng IUncHons sccscecsussessssisunesadesinssaxseavasistagosatodsguebedecdsseaccsivesedensssadsacsoubeasys 21 2 3 2 Rettieving PUnelons iuiiecs eda eo D dln Bia erue r E dan Rese a doe ieee 27 4A Se LORIE FUN CU ONS MN E 36 B ANNEX 1 DISK DATA ACC SS icsisisssisssisscisainivannnsscnensne
60. k Q void xf tree cleanup parser long fd long error Releases the memory resources taken by the XML parser for all the open files QO void xf tree cleanup all parser Gets the name of the current element This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission d ei m O S Critical EE XML Binary CFI ie ENGENHARIA J alias File Handling 8 ANNEX 4 SELF TEST CASES 8 1 test_binxml Code Page SO UM DME L1PP 0005 02 12 08 2 5 48 of 48 The BinXML FH library provides a test program to check if it is everything correct with the installation of the library itself and with the dependent libraries The test program is located in test bin test binxml and the source code in test src test binxml c The main goal of this program is to use all the set and get functions available it produces a hybrid file data verify eef 58009 bytes in test data directory For testing the reading of variable arrays it uses another hybrid file var array EEF 757 bytes located in the same directory The values used are the following Short 300 Int 3000 Float 350 543 Long 3500768 Double 3500 10 Character d Byte 100 String Test binxml fh Complex R 0 2 I 0 6 Doublecomplex R 0 258 I 0 636 Short_vector 567 Int_vector 2345 Byte_vector 86 Long_vector 6745662 Float_vector 54637 8
61. l of this handler is to provide to the API an efficient way to handle binary data files using XML Schemas To respond to this demand the Object Handler uses the BinX library that makes possible to load in memory the content of binary files or writing data into them These two operations are done based on a BinX language file that describes the binary data structure and allows BinX to translate it into an XML Schema This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J software File Handling BinX XML Schema BinX Library Object Handler Figure 1 Accessing data variables The Object Handler uses the BinX API functions to load the data into memory thus creating objects that represent that information When the objects are created a small table identifying the new objects is filled with the object and an object identifier an integer By this approach it shall be very easy for the API to know the available objects and even to destroy those that are no longer need The Object Handler management table will have the following structure The ObjectID field identifies in a unique way a memory loaded object To describe better the role of Object Handler let us consider that binary data file with several structures is going to be loaded in memory using BinX Imagine that the user wants
62. les into header and data block or join them into a hybrid file To split a hybrid file the test must run with the following arguments e test bin file transform binxml config xml split data data_verify eef To join a header file with a datablock the command is the following e test bin file transform binxml config xml join data data_verify eef This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Q nnns Date 02 12 08 deim s Li EEXML Binary CFI is ENGENHARIA J software File Handling e B 9 ANNEX 5 NESTED VARIABLE ARRAYS The support for nested variable arrays is a new functionality that is available in release 3 4 of BinXML File Handing 9 1 Purpose Since the introduction of variable arrays support in this tool it has been studied the possibility of migrating the schemas definition to contain exclusively variable arrays which leads to the implementation of nested variable arrays By definition a variable array is a structure that uses information read from the binary file to define the actual size of the array Introducing the variable arrays on the dataset definition means that part of the information relevant to the Dataset Size will be inside the Dataset itself Let s see an example with the proposed changes to the schemas Considering the following schema in the current format lt binx gt lt definitions gt lt
63. lex int object ID char varname tVectorDoublecomplex v doublecomplex This function receives an identification of an object already loaded object ID and set the variable varname with the content of the double complex vector v doublecomplex The type of this vector is tVectorDoublecomplex and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 20 set matrix int Syntax EKerou Selimatrixeintk ant iobyectalbD ehan Avarname tMatrixinkg imat rix nE This function receives an identification of an object already loaded object ID and set the variable varname with the content of the int two dimension array matrix int The type of this vector is tMatrixInt and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 21 set matrix float Syntax tError set matrix float int object ID char varname tMatrixFloat matrix float This function receives an identification of an object already loaded object ID and set the variable varname with the content of the float two dimension array matrix float The type of this vector is Mat rixFloat and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 22 set matrix long Syntax tError set matrix long int object ID
64. loat value tError set double int object ID char varname double value tError set complex int object ID char varname tComplex value tError set doublecomplex int object ID char varname tDoubleComplex value tError set vector byte int object ID char varname tVectorChar v char tError set vector short int object ID char varname tVectorShort v short tError set vector int int object ID char varname tVectorInt v int tError set vector long int object ID char varname tVectorLong v long tError set vector float int object ID char varname tVectorFloat v float tError set vector double int object ID char varname tVectorDouble v double tError set vector doublecomplex int object ID char varname tVectorDoublecomplex v doublecomplex tError set matrix ineine ODJEece ID char Weenies EMatrixint Matrix int tError set matrix float int object ID char varname tMatrixFloat matrix float tError set matrix long int object ID char varname tMatrixLong matrix long tError set matrix double int object ID char varname tMatrixDouble matrix double tError set matrix byte int object ID char varname tMatrixByte matrix byte tError set matrix doublecomplex int object ID char varname tMatrixDoublecomplex matrix dcomplex void readNestedData char xml fname void objectID char getDataFormat void objectID void writeNestedData void data char xml fname char fname void freeNestedData
65. mory structure or schema file which implies a recompilation of user s code 2 Writing data 2 1 Writing the data is done always according to the specified format Any mistake in the format description may cause a segmentation fault since that is the only way that BinXML FH can know the format of the memory structure 3 Reading data 3 1 The data read from the BinX file always follows the same approach to produce the memory structure which means that the BinXML user s memory structure must be defined accordingly 9 3 Approach In C there is a way of knowing the class of a specific object and in order to mimic this functionality in C it is necessary to create a workaround To inform BinXML about the format of the User s data structure it is necessary to use a pseudo language that can be used to validate the schema data types against the User s defined memory structure The pseudo language has the following element B S L L F Represent the basic types byte short int long float and double and D 0 9 Represent the number of elements of a BinX Fixed Array and correspond to a static allocation in the memory structure i e array 4 for example and Are used to enclosure a repeating block which is typically an array element are used to enclosure a fixed size element structure and a variable size structure i e a structure that contains variable sized arrays or linked lis
66. not defined ERROR FINDING STRING This code is used when a string is no found in a file or inside another string C ERROR READING FILE This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d ei mos Critical EE AM anay El uf ris ENGENHARIA J software File Handling T Error used to inform that the requested file could not be opened either because it doesn t exist or it hasn t the proper permissions ERROR REMOVING OBJECT This error occurs when the provided ob ject ID doesn t exist Q TERMINATE This code is used to inform that a termination or re initialization of the library is required for example when the default configuration file is created using the function create config file ERROR WRITING FILE Error used to inform that the requested file could not be written into either because it doesn t exist or it hasn t the proper permissions 4 2 2 eErrorLevel Error Level There are 3 levels of errors 1 ERR LEV LOW is the lowest of all and usually is associated with errors that doesn t affect the library too much It works like a warning 2 ERR LEV MEDIUM is the intermediate level The program can follow but some action must be taken in order to not break the program 3 ERR LEV HIGH is the highest level and it usually identifies an error that will require the termination the p
67. nspansncnancscnmsusasaiaianoissanaseanseassshuaencianannbeion 44 6 ANNEX 2 BINX OBJECT DESCRIPTION Gs dene eHaaeRkR ARR RAN RRRE KROR ER KRARRRE EAR eRKRRNAR REIR EROR KEARRR ERR DERE ERR 45 GT USER TV AC 45 6 2 Used Methods S 45 7 ANNEX 3 EARTH EXPLORER File Handling CFI e seeseoscesseeseeseescosscesosscsscoscessossosscessessese 47 TA Methods Used 47 SANNEX 4 Self Iosl CASES eine ente EHE NENNE vv eni 48 9 1 test DINAM t 48 8 2 test oo 51 8 3 file transfor 52 9 ANNEX5 Nested Variable Array Sasnnn E 53 JL C cucoana A 53 mel 56 ED 3 ADDEOSCI S o OPERE E actos be RERO E E E E EA 56 List of Figures Figure 1 Accessmp data variables iosseerdenseret orco terre ee X eb E e EEE EEE epe cepas EA kou tese o eene M epe tes 4 Figure 2 Directory stricture ot the DOCI esee Poeti bec Se Pee eb De ee inlaw Le S e Du ba d dE 6 Figure 3 Retrieving a scene from a hybrid XML data Dile uo iioi est ipae PRESS gb re eo Pap R bo EKE 7 Figure 4 Example of how to use the Library to retrieve information eese 8 Figure 5 An example of how to use the Library to save inf
68. ntified by object ID Then the value is converted to short and is returned to the user in value The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 7 get int Syntax tError get int int object ID char varname int value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to int and is returned to the user in value The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 8 get long Syntax tError get long int object ID char varname long long value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to long long C99 Standard and is returned to the user in value The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 9 get float Syntax tError get float int object ID char varname float value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to float and is returned to the
69. nts according to block indices from a hybrid XML file of name data filename and stores it into a temporary binary file whose name is returned inbin filename The function returns a tError structure to inform the status of the operation The data element that is extracted is related with the binary data block identified by parent name This section is described in a structure in the header that contains the block name the offset where the binary data block starts the size of each element of the block and the number of elements existing in that block see 2 6 Hybrid file header NOTE This function allocates memory for returning the binary file name and the user must release it when it is not used anymore The errors returned are e ERR CODE ERROR ALLOCATING MEMORY e ERR CODE ERROR FINDING STRING e ERR CODE ERROR READING FILE e ERR CODE ERROR WRITING FILE e ERR CODE NO ERROR 4 4 1 10 prepare schema Syntax LEETOr prepare schema char fname bin char fname_template tiIndices var_indices chat fname schema char endian This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 7 Code d 1 mos Critical EE XML Binary CFI nd 02 bs H ENGEN ARIA J software File Handling page ae This function uses a schema template with file name name template which describes one element of data to create another schema containing a specifi
70. nxml tests the functionality with primitive BinX data types and with arrays of primitive types The test program binary is test bin test nested which corresponds to the test src test nested c source file The main purpose of this test is verifying that nested data structures of different complexity and endianness can be read from and written to disk If both read and write operations complete without errors the files are compared If they have no differences the test is considered successful The program uses XML and data files DBL nested5 b nested5 1l nested6 b nested6 1 and nested7 all of them located in the test data directory nested5 b dbl and nested6_b dbl have data in big endian byte ordering while the others have it in little endian byte ordering A nestedout dbl is created in the data directory during the execution If we consider the BinX data to be arranged in a tree like structure with complex data types as arrays at higher levels and primitive data types at lower levels the data structures used by test nested have arrays at three levels The important differences between them are the array types fixed or variable at each level The data structure of nested5 b and nested5 1l files has a variable array at the highest level another variable array at the second level and a fixed array at the third level The data structure of nested6 b and nested6 1 files differs because it has a fixed array at the highest level The
71. ocates all the memory that is necessary to represent the information retrieved from the BinX file and returns a pointer to that memory structure For more details see Annex 5 4 4 2 29 getDataFormat Syntax char getDataFormat void objectID This function is used to retrieve the string that describes the memory structure using a Pseudo Language based on the BinX schema that describes the binary file It receives a void objectID that corresponds to the Object that was used to read the BinX schema which means that this function ALWAYS require an previous invocation of readNestedData to get the object ID For more details see Annex 5 4 4 3 Storing Functions An example of how can these functions be used to save information is shown in Figure 5 4 4 3 1 save data file Syntax tError save data file int objectID char bin filename This function saves to a binary file named bin filename the information contained in an object of BinX identified by object ID It Returns one of the enumerated t Error values according to the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 2 set bit for future implementation Syntax igneus xu deuhe Gute lee ID Cleve t varname iEEUSIEUBaRE Star EBIC asm pit ie ehar valus This function is used to set a bit at position bit nr of value in variable varname according to value that can be
72. oftware File Handling Page 44 of 44 5 ANNEX 1 DISK DATA ACCESS In order to select the best way to access data from a hybrid XML file a performance test was created by defining a binary file which contains 4000 scenes and a program was written to perform two different methods of accessing data In method A an XML schema is created containing the 4000 schemas of the scenes and the binary file is read into memory using this huge XML schema which means that the binary file is completely loaded into memory Then each one of the N scenes is accessed reading a value of that scene In method B a chunk of data corresponding to one of the N scenes is extracted into another binary file and afterward loaded into memory using the XML schema of the scene Then a value of that scene is read from memory In this test each scene was comprised of a total of 11 Kbytes and the test results are shown in Table 5 11 1 7 0 015 1100 1 72 0 325 5500 1 72 LJ 11000 1 74 34 22000 2 21 6 36 40000 5 12 9 Table 5 Time used accessing a variable of N scenes in seconds The machine used to perform the tests has the following characteristics CPU Intel Pentium 4 2 4 GHz RAM 470 MB DDRAM 333MHz Disk IDE 36 GB UDMA 100 I O 50 39 MB s From this analysis the conclusion obtained was that there should be two ways to prepare the binary file and schema for read access depending on the size of the file handled The BinXML FH provides a default au
73. ormation esee 9 Figure 6 Din X data Stricture iieri eie aa A lauded Ri OR eee te ERRORS De dU 45 Figure 7 Retrieving data from a hybrid XML data file using the variable array approach 35 Figure 8 Generating the binary file using variable array approach serene 39 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 Code d m m Critical EE XML Binary CFI Date 02 12 08 ENGEN ARIA J software File Handling Eus an Page vi List of Tables Table 1 Table Of JAGtolyImsisinis eios e p pe alice Ere Ende Ea Pr eae EUER uM ee EDU Se XA Ie E UD EE br DEDE ea 1 Table 2 Applicable Documents asi ccisszeseccatisertstetetsstuseivanenatesuepeticastehatcagenatolasbesiedccubatsdtphada Deb o SAN eU Pepe Da 1 Table 3 Reference DOCUI ehls iani Ert Ere XXE FRE XARER GER EAM ER canaanbe cddans sacenaes saadsaaheacansshobecuasedbasasseaccana 2 Table 4 Library dependetieles uua keiten e yere Ert Masui sends ins PET ch Pe ca FoU rv cnra Pelea tU une 5 Table 5 Time used accessing a variable of N scenes 1n seconds retento etre ere tea a 44 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 A Date 02 12 08 de imsEEH mere me EHOGENHARIA J software File Handling Page 1 of 1 1 INTRODUCTIO
74. plex value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to double complex and is returned to the user in value The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 13 get struct under development Syntax tError get struct ine Object ID Char vanname Void Struct varn St Ste SI Ze This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ipis ENGENHARIA J software File Handling Page NGA This function returns a pointer to the structure associated to the variable identified in the schema by varname This structure is defined on the schemas but must be known by the library when invoked The function returns a tError structure to inform the status of the operation SPECIAL NOTE As opposed to the previous access methods the structure accessed through this one may not change in runtime i e altering the BinX schema as the structure NEEDS to be known before hand to the function invoking the CFI Any change in the structure format will ALWAYS require recompilation to avoid errors not in the CFI but in the functions using it The errors returned are e ERR CODE ERROR ACCESSING
75. rogram or the reloading of the library s configurations 4 2 3 eStartBit Start Bit It can be MSB or LSB and is used to identify the starting bit of the counting Example Considering the byte 10101110 and that the left bit is the more significant bit the get bit MSB 1 will return the 2 bit counting from left 0 while get_bit LSB 1 will return the 2 pit counting from right 1 4 3 Structures Types Q tError Structure used to contain the error information Composed by ErrorCode and ErrorLevel enumerated plus an array of characters to incorporate additional error information LJ tDataBlockInfo Structure containing the information of any Measurement Data Set Descriptor in an XML ASCII header of any hybrid product file It is used to divide the product file in data blocks for advanced access Q tComplex This structure is used to represent the complex values composed by two float values This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 e hae Date 02 12 08 d eim As Critical EE XML Binary CF Pe b ENGENHARIA J TT File Handling Page 20 of 20 QO tDoubleComplex This structure is used to represent the complex values composed by two double values U tVector This structure is used to represent a one dimensional array of unknown type with the additional information of the number of
76. s tMatrixByte and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 25 set matrix doublecomplex Syntax tError set matrix int int object ID char varname tMatrixDoublecomplex matrix dcomplex This function receives an identification of an object already loaded object ID and set the variable varname with the content of the tDoubleComplex two dimension array matrix dcomplex The type of this vector is Mat rixDoublecomplex and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 26 writeNestedData Syntax This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 Q nones Date 02 12 08 deim s llt EE XML Binary CFI Pe E ENGENHARIA Jr software File Handling T ere void writeNestedData void data char xml fname char fname This function is used to write to a binary file name the memory structure data described in the BinX schema xml fname Its main intent is to write structures with nested variable arrays For more details see Annex 5 This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d ei mos Critical EE XML Binary CFI die i m ENGENHARIA Jr s
77. t Represent a pointer Is associated with variable arrays and represent a number that must be read from the buffer When is found on the format string it appears always in the sequence P where can be one of B S I or L and represent the type of the number that is used to represent the array counter This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission C deim os lilii ENGEN ARIA d Rules software Code SO UM DME L1PP 0005 EE XML Binary CFI Pte 02 12 08 Issue 2 5 File Handling Pipe Td A Variable array is always translated in a linked list memory structure and vice versa which means that always after the description of the Variable Array Element structure a new pointer must be inserted to point to the next element of the Variable Array Therefore to describe a Variable Array of integers with a counter defined by a Byte it should be used a format BP IP If instead of an integer the array element is a complex structure let say a fixed array of 3 floats the format should be BP 3 F P To describing a fixed array of 4 elements each one being a variable array of Integers with a counter defined by a short the format should be 4 SP IP Example Considering the following user memory structure and the corresponding BinX schema typedef struct data float valf Ime vali double vald Si PUG aata
78. te packed tPolData lt defineType typeName tPolFixArray gt SSthuctb typedef struct s_VarArrayFixedType lt arrayFixed varName PolFixArray byteOrder littleEndian gt tPolData polFixArray 2 lt str ct gt struct s_VarArrayFixedType next lt byte 8 varName polID byteOrder littleEndian gt __attribute__ packed tVarArrayFixedType integer 32 varName polVal byteOrder littleEndian struct typedef struct s dstype dim indexTo 1 gt lt dim gt char charElement arrayFixed short varArrayFixedTypeCount lt struct gt tVarArrayFixedType varArrayFixedType lt defineType gt short shortElement lt defineType typeName dstype gt struct s_dstype next lee Yee attribute packed dstype n5 lt byte 8 varName charElement byteOrder littleEndian gt typedef struct byte count dstype_n5 dataset attribute__ packed tProduct ns lt arrayVariable varName varArrayFixedType byteOrder littleEndian lt sizeRef gt lt short 16 byteOrder littleEndian gt lt sizeRef gt lt useType typeName tPolFixArray varName typedFixArrayElement byteOrder littleEndian gt lt dim gt lt arrayVariable gt lt short 16 varName shortElement byteOrder littleEndian gt lt struct gt lt defineType gt lt definitions gt dataset src nested5_1 db1 byteOrder littleEndian gt lt arrayVariable varName dsarray byteOrder littleEndian gt siz
79. the library to write files and to give him quota enough to deal with the temporary files created while using the library either for reading and writing The worst case estimated quota that it shall be necessary according to the files that the user is going to process shall be double the size of the processed files to allow complete rewriting of each file into a temporary directory For accessing values defined in the ASCII part of the hybrid XML the header the libray shall use the EEFH RD 4 as already defined providing access to it to the user as well through the same interface The library shall also provide access to the Binary part of the hybrid XML file through functions described in subsequent chapters To explain how the library works let us consider a SMOS L1 applicable scenario where it exists a block of data called Scene and a hybrid xml file LO Product containing a specific number of scenes The header of the hybrid xml contains a tag with the number of scenes within the file and another tag explaining the offset where the binary data starts in the hybrid xml file This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 7 Code SO UM DME L1PP 0005 d e 1 m S EE XML Binary CFI pate 3 OEE Oe A Issue 2 5 ENGENHARIA J rM File Handling TE sts L1PP l E s L maoec N Read Access LIPP Core functions M Am ooo epe e qe
80. the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission 78 Code SO UM DME L1PP 0005 d e 1 m O S EE XML Binary CFI Date 04 14 08 A Issue 2 5 ENGENHARIA Ji software File Handling Page 34 of 34 4 4 2 22 get_vector_doublecomplex Syntax tError get vector doublecomplex int object ID char varname tVectorDoublecomplex array dcomplex di This function is used to retrieve an array of tDoubleComplex and the number of elements in that array contained in the variable with name varname associated with the object identified by object_ID The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 23 get matrix int Syntax tError get matrix int int object ID chat varmame tMatrixInt matrix int yy This function is used to retrieve a two dimension array of int and the number of elements in that array contained in the variable with name varname associated with the object identified by ob ect I The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 24 get matrix byte Syntax tError get matrix byte int object ID char varname tMatrix
81. tion 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 14 set vector int Syntax Error set Ves objece TID ehar b ranmamcemgtaves orto This function receives an identification of an object already loaded object_ID and set the variable varname with the content of the int vector v_int The type of this vector is tVectorInt and his definition is described in the section 3 3 The errors returned are This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA Uu software File Handling Page 40 of 40 e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 15 set vector long Syntax tError set vector long int object ID char varname tVectorLong v long This function receives an identification of an object already loaded object ID and set the variable varname with the content of the long vector v long The type of this vector is tVectorLong and his definition is described in the section 3 3 The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 3 16 set vector float Syntax tError set vector float int object ID char varname tVectorFloat v float This function receives an identification of an object already loaded object ID and set the variable varname with the cont
82. to mode which allows it to choose which method is best suited for the task requested and it also provides a manual mode which allows the user to choose which method to use All this is still under development This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind ipis ENGENHARIA J om File Handling paget 45 of 45 6 ANNEX 2 BINX OBJECT DESCRIPTION This library use BinX API to handle binary data files by defining XMLSchemas describing the content of these files In this section it is described the objects and methods of BinX that are used BxBinxFileReader BxDataObject BxDataset BxUnsignedByte amp BxUnsignedShort16 BxUnsignedInteger32 BxUnsignedLong64 BxArray Figure 6 BinX data structure 6 1 Used Types BxBinxFileReader Read and parse a Binx XML file BxDataset Ordered collection of objects of several BinX data types BxDataObject Class that implements all BinX data types BxUnsignedByte8 BinX data type 8 bit unsigned byte BxUnsignedShort16 BinX data type 16 bit unsigned integer BxUnsignedInteger32 BinX data type 32 bit unsigned integer BxUnsignedLong64 BinX data type 64 bit unsigned long OoUDUeOD ee eo BxArray Base class for all BinX array types 6 2 Used Methods Q BxArray get int index Get a copy of a single data element from an array
83. user in value This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission SO UM DME L1PP 0005 ao Code d 1 mods Critical EE XML Binary CFI 02 n H ENGEN ARIA J rem File Handling TE 30 of 30 The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 10 get double Syntax tError get double int object ID char varname double value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to double and is returned to the user in value The function returns a tError structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 11 get complex Syntax tError get complex int object ID char varname tComplex value This function requests the value of the variable identified by varname in the object of BinX identified by object ID Then the value is converted to complex and is returned to the user in value The function returns a tEr ror structure to inform the status of the operation The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR 4 4 2 12 get doublecomplex Syntax tError get doublecomplex int object ID char varname tDoubleCom
84. value The memory pointed by value contains the values read from the file according to the schema It is the responsibility of the caller to cast the void in value to the correct structure and any change in the schema related with the variable array must be taken into account on the caller s structure The function returns a tEr ror structure to inform the status of the operation This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI ind icis ENGENHARIA J m File Handling oe ae SPECIAL NOTE As opposed to the previous access methods the structure accessed through this one may not change in runtime i e altering the BinX schema as the structure NEEDS to be known before hand to the function invoking the CFI Any change in the structure format will ALWAYS require recompilation to avoid errors not in the CFI but in the functions using it The errors returned are e ERR CODE ERROR ACCESSING OBJECT e ERR CODE NO ERROR ERR CODE INVALID DATA TYPE e ERR CODE ERROR ALLOCATING MEMORY 4 4 2 16 get vector byte Syntax tError get vector byte int object ID char varname tVectorChar array char This function is used to retrieve an array of char and the number of elements in that array contained in the variable with name varname associated with the object identified by object ID The function
85. y File filename BxByteOrder Write the application data value of each of the dataset member objects to a specified binary file with the specified byte order This document is property of DEIMOS Engenharia and cannot be distributed or duplicated without its written permission Code SO UM DME L1PP 0005 d e 1 mos Critical EE XML Binary CFI d ll ENGENHARIA software File Handling Page 47 of 47 7 ANNEX 3 EARTH EXPLORER FILE HANDLING CFI The BinXML FH library uses the EECFI to access the ASCII XML information In this section it is described the functions that are used 7 1 Methods Used Q void xf tree init parser char file long error Loads an XML file into memory Q void xf tree read string element value long fd char element char value long error Reads a string value Q void xf tree read string attribute long fd char element char attribute name char attribute value long error Reads an attribute as string Q void xf tree add child long id char parent char name long error Add a new element to an XML document as a child of parent Q void xf tree add attribute long id char current char name long error Add a new attribute carried by an element Q void xf tree set string node value long id char name char value char format long error Set a string node value Q void xf tree write long id char name long error Write the data to a file on dis

User Manual - SMOS L1 Processor Prototype

Contents

Download Pdf Manuals

Related Search

Related Contents