Home

sound management integration

1. printfhnNX1 Record Sound print nNN2 Play Sound printf NnNNO Exit n printf Nn tEnter vour choice number answer getchar 75 while a getchar n printf nYour answer is c n answer switch answer case l clr_scr printf t RECORD SOUND WV SSantex_record filename sfilename a getchar break case 2 clr_scr printf Wt PLAY SOUND WM puts Enter the file name to play user defined gets sfilename SSantex_play sfilename a getchar break case 0 cir scr printf Ninni gya akak ak ak a a k ak ak ake aale ak ak ak ak ake aak akak ak printf NNiv THANK YOU n printf n WNt HAVE A NICE DAY M print NOLES ASSES ark Na Sy break end switch end while ans not in 0 3 getchar end while ans not 0 end main 76 fee deck hehe bebe oe e je e ke be be hehe cioe eee RPLAY C ake akk ak ke ke ae je akk ake oj he afe ake le e je ake ake oe oe ak ake ok ake ak oe ake joke EK KK Author Yavuz Vural ATILA Rank LTJG Turkish Navy Advisor Prof Vincent Y LUM Date 27 August 1990 Revised 17 September 1990 Description This program is designed for the ANTEX VP620E to play the previously recorded sound
2. gt NAVAL POSTGRADUATE SCHOOL 2 Monterey California A245 774 RU gt y FEBLL B i F X o ss s THESIS DESIGN AND IMPLEMENTATION OF A MULTIMEDIA DBMS SOUND MANAGEMENT INTEGRATION by Yavuz Vural Atila December 1990 Thesis Advisor Vincent Y Lum Approved for public release distribution is unlimited 92 02880 m LLLI UNCLASSIFIED SECURITY CLASSIFICATION OF THIS PAGE REPORT DOCUMENT ATION PAGE Approved for public release distribution is unlimited a NAME OF MONITORING ORGANIZA Computer Science Dept ur scu Naval Postgraduate School Naval Postgraduate School 6c ADDRESS City State and ZIP Code 7b ADDRESS City State and ZIP Code Monterey CA 93943 5000 Monterey CA 93943 5000 8a O UNDING SPONSORIN 8b YMRO ORGANIZATION if applicable 8c ADDRESS City State and ZIP Code ACCESSION NO 11 TITLE Include Security Classification Design and Implementation of a Multimedia DBMS Sound Management Integration PERSONA FIO Atila Yavuz Vural 3a OF REPO 13 14 DATE OF REPORT Year Month Day 1 Master 5 Thesis I RH 112 89 1o 12 90 December 1990 13 1 ON The views pelle in this thesis are those of the authors and do not reflect the official policy or position of the Department of Defense or the U S Government COSATI CODES 18 SUBJECT TERMS Continue on reverse if necessary and identify by block number 5 a
3. 102 INITIAL DISTRIBUTION LIST Defense Technical Information Center Cameron Station Alexandria Virginia 22304 6145 Library Code 52 Naval Postgraduate School Monterey California 93943 5100 Center for Naval Analysis 4401 Ford Ave Alexandria Virginia 22302 0268 John Maynard Code 042 Command and Control Departments Naval Ocean Systems Center San Diego California 92152 Dr Sherman Gee ONT 221 Chief of Naval Research 880 N Quincy Street Arlington Virginia 22217 5000 Leah Wong Code 443 Command and Control Departments Naval Ocean Systems Center San Diego California 92152 Professor Vincent Y Lum Code CsLm Department of Computer Science Naval Postgraduate School Monterey California 93943 10 14 Deniz Kuvvetleri Komutanligi Personel Daire Bagkanligi Bakanliklar Ankara TURKEY Deniz Harp Okulu Komutanligi 81704 Tuzla Istanbul TURKEY Yavuz Vural ATILA Dz Kd Utgm Selami Ali Mahallesi Yeni Dershane Sokak 8 3 81140 Uskudar Istanbul TURKEY Professor David Hsiao Code CsHq Department of Computer Science Naval Postgraduate School Monterey California 93943 Professor C Thomas Wu Code CsWq Department of Computer Science Naval Postgraduate School Monterey California 93943 Professor Klaus Meyer Wegener University of Erlangen Nuernberg IMMD VI Martensstr 3 8250 Erlangen GERMANY Dr Bernhard Holtkamp University of Dortmund Department of Compu
4. 51 B SOUND MANAGEMENT INTEGRATION INTO MDBMS 55 VI SUMMARY AND CONCLUSIONS 58 As REVIEW OF THESIS bez eee Fs ates poto 58 B APPLICATION AREAS 60 C FUTURE WORKING 61 LIST OF REFERENCES UT red HS Ue BSE Dag BARS 62 APPENDIX A PC SOUND MANAGEMENT PROGRAM CODE 65 APPENDIX B SUN SOUND MANAGEMENT PROGRAM CODE 97 INITIAL DISTRIBUTION LIST 2 ee Ge ee Re EE eR 103 LIST OF FIGURES Figure 1 The Architecture of the 5 7 Figure 2 Structure of a Media Object SOUND 8 Figure 3 The MDBMS System 12 Figure 4 The main menu of MDBMS 15 Figure 5 The image of USS Kitty Hawk 21 Figure 6 MDBMS Table Array and Table List catalog tables 27 Figure 7 MDBMS Att Array 28 Figure 8 MDBMS _ 29 Figure 9 The Image and Sound Relational Tables 32 Figure 10 The Sound Management in IBM compatiblePC 39 Figure 11 The main menu of PC Sound Management 44 Figure 12 The main menu of the SUN Sound Management 48 vi ACKNOWLEDGEMENTS I am grateful to Dr Klaus Meye
5. queries the MDBMS system itself creates the formal SQL statements and executes the requested operations After running the MDBMS program which is written in C language the user gets the main menu as shown in Figure 4 First we will try to explain each option in numerical order and then show the application of interface with a couple of examples in the next section Multimedia Database Management System Create Table Insert Tuple Retrieve Delete Modify Quit Figure 4 The main menu of MDBMS The first option is the create table operation which is used to create relational tables in the MDBMS It allows a user to enter the table name and then the attributes which have data types like character integer float and media types image and sound The program displays a warning message in case of table name duplication After inserting all the data about the new table attributes and their data types the user has an 15 opportunity to modify the entered data and insert more attributes or delete already inserted attributes The insert tuple operation allows the user to enter tuples one at a time When the user enters the table name to insert tuple the process starts and the system asks the user to enter interactively the values for the attributes in the given table in the sequence according to the order defined in the relational table For media data which should be transferred to the MDBMS environment befo
6. struct sockaddr amp server sizeof server 0 perror connecting stream socket 98 exit 1 if write sock filename 12 lt 0 gets the filename for playing perror Writing on stream socket close sock printf n NPlease press ENTER to continue return PE A e eak ak k ae e ake ae e ie ade ake e ake ake e ake ak SOUND INFO READY 8 ak ak ak ak ak ae ae ak ake ake akk kk This part reads the sound file header information from the given sound text file which is already sent from PC to SUN aaao e oko ak ae III ae ake ake I ae ake ake ake ake ake ake ahe ak ake ahe ake ake ake ake ake ake ake ak ae ak ak Snd_inforead filename r char filename given input text file struct SND_HDR r FILE f if f fopen filename r NULL open for reading perror Cannot open file for input retum SOUND ERROR read the header from the given input text file fscanf f 96s r sfname fscanf f 26d amp r s size fscanf f Tod amp r s samplrate fscanf f 95d amp r s encoding fscanf f f amp r gt s_duration fscanf f d amp r gt s_resolution fclose f return f She de obe ae ele MAIN Neo ke oe ke ok ak ak ak ak ae ke ak ke ake ake ake ak ak ak ak ake ake ok a ake ak ak ak ak ak k kk main char filename 13 char answer a char pcname 7 s
7. newname temp snd VP620 amp vpfunction amp monitor amp samplerate newname vpfunction START vpstart VP620 amp vpfunction printf nRecording in progress Press lt lt SPACE BAR gt gt to stop n n vpfunction STATUS vpstatus do VP620 amp vpfunction amp overload amp hundsec amp sec amp error amp state printf State d Error d Seconds d 02d Overload d r state error sec hundsec overload while kbhit amp state 3 These statements always required to close the file vpfunction STOP vpstop VP620 amp vpfunction print n print v NaEnd of recording session n vpfunction END vpend 70 VP620 amp vpfunction Update the header fields now that the file has been recorded sz FileSize newname sizeof struct SND HDR hdr s size sz if sainplerate 0 srate 8000 else srate 16000 hdr s samplrate srate hdr s encoding sencode ADPCM code sdur sec float hundsec 100 hdr s duration sdur if sencode 1 hdr s resolution 8 set bits per sec else hdr s resolution 0 header values of recorded sound file printf nRegistration Data Values of the Recorded Sound File An printf nFILENAME s n filename printf NnNaNSIZE 91 bytesw hdr s size printf NSAMPL
8. amp vpfunction amp srate filename open file vpfunction STATUS printf n tPlease press lt lt SPACE BAR gt gt to stop playing NN do VP620 amp vpfunction amp overload amp hundsec amp sec amp error amp state printf State d Error d Sec d d Overload d N state error sec hundsec overload while kbhit amp state 3 printf n n n END OF PL A Y printf n nPress ENTER to continue these statements always required to close the file vpfunction STOP VP620 amp vpfunction vpfunction END VP620 amp vpfunction else return ERROR FREE displayerr READ file read error return SOUND ERROR de ake ake ae sje ak ae afe ake ok ae kk CLEAR SCREEN ake ake ok ake be ak ak be ok ak ake ake ok kk oe oe ok akk akc afk oke ak ak ak ak clr scr putchar 033 putchar putchar H putchar 033 putchar putchar J P be eae e kk ake ae ake i ake ake ake ake ake ake ak ak ake M AIN aake ake ake ake ake ake ake ake ake ake ak ak ake ake ake ake do a ke ake ake ake ake a te main char filename NAME LENGTH char sfilename NAME LENGTH char answer a while answer 0 answer while answer lt 0 Il answer gt 2 clr_scr printf Wn n ANPC SOUND MANAGEMENT SYSTEM printf N
9. int state error sec hundsec overload int monitor 1 record monitor always on long int sz int srate sencode sresol float sdur ERROR err struct SND HDR hdr if read snd hdr filename amp hdr 0 read the sound file header vpfunction BEGIN alert to the driver port 0x280 useint 2 VP620 amp vpfunction amp useint amp port if hdr s samplrate 8000 srate 0 else srate 1 vpfunction PLAY VP620 amp vpfunction amp srate filename ope file vpfunction STATUS do VP620 amp vpfunction amp overload amp hundsec amp sec amp error amp state while kbhit amp state 3 these statements always required to close the file vpfunction STOP VP620 amp vpfunction vpfunction END VP620 amp vpfunction 86 else displayerr READ ake ake ake ake je e de de e de de ake ode obe be ae ake e ake ake ae akk kk kk MA IN Mc ake be ake je be bc e de be ak aie ok ak oj kc main rgc argv receives sound ije to play int argc char argv SSantex_play argv 1 end main 3c ak oko oe kc OR ORO OR okt ak ak ok kok ak ok ak kk ok kok i ak ak kk ok kk ak kk iC kk ak ak ak ak ak ai oe ake ak ake ake ak a ake akc ac afc akc e ahe e ae ake ake ake ake ahe ake ake ae SND STRU C ak ak ae ae ak kk ae ke ek ake ake a ake ake ake a ake ak ak T
10. EN89 Let us illustrate this by assuming that in addition to the SHIP relation we have created a relational table OFFICER with the attributes O ID O NAME RANK SALARY PHOTO and VOICE We will use CAPT ID attribute in SHIP table and O ID attribute in OFFICER table for the join condition Now let s 24 assume that the user wants to retrieve the ship names officer names ranks photos and voices for the officers whose rank is Lt Cur and has soft voice in description of sound For this retrieval the query in extended SQL will look as follows SELECT 5 name OFFICER o name OFFICER rank OFFICER photo OFFICER voice FROM SHIP OFFICER WHERE OFFICER rank Lt Cdr and CONTAIN OFFICER voice soft voice and SHIP capt id OFFICER o id Again as before in this system the user enters this query interactively after selecting the third operation from the main menu The session will appear as follows Select the table s separate by comma lt gt ship officer cr Please enter the join condition ship capt id officer o id cr Table Ship Select the attribute s separate by comma lt gt 5 name cr 1 Table Officer Select the attribute s separate by comma name rank photo voice cr 4 ship s name s name c20 officer o name o name c20 officer rank rank c20 officer photo photo image 25 officer voice voice sound Any condition y n y cr Grou
11. data From this point of view the big effort concentrates on the storage and retrieval of the multimedia data by the content of the data which is processed Handling content search in multimedia data is a difficult problem one needs to find ways to handle a very large amount of multimedia data and to search and find the appropriate data conveniently and efficiently based on the contents of the media Without the ability to do content search having a MDBMS would be like having a DBMS that cannot process queries on standard data The content search issue on media data is not possible with the current methods used on formatted data structure so we need to use an abstract data type concept LM89 Image sound signal text and graphic data will be treated as new data types Any attribute of an object can have one of these data types instead of the usual data types like characters integer boolean etc We also need to define operations to process create retrieve update delete these new media data values The MDBMS Project in the Computer Science Department of the Naval Postgraduate School was formed to develop a technology that would allow us to handle multimedia data as conveniently as we can process the standard data with the emphasis of providing content search capability WK87 LM89 Besides the MDBMS Project at Naval Postgraduate School there are a number of researches going on in multimedia data processing around the world Among
12. default unknown argument 94 errexit unknown argument break end of switch end of if Set up the socket to receive data socket sin htons port fdw filefd Make a socket call if fdr socket SOCK STREAM struct sockproto 0 amp recv socket SO ACCEPTCONN SO KEEPALIVE lt 0 errexit socket fprintf stderr socket fd d n fdr Accept a connection fprintf stderr posting accept fd d n fdr if accept fdr amp send socket lt 0 errexit accept fprintf stderr accepted connect from partner Read in the data from the socket and display it on the screen or write it to the file if netcnt soread fdr buf bufsize lt 0 errexit soread fprintf stderr read d bytes n netcnt if cnt write fdw buf netcnt lt 0 errexit write if fdw gt fileno stderr close fdw soclose fdr errexit errstring char errstring if errno experror errstring else fprintf stderr s nusage receiver file name zbuffer size n errstring 95 if fdr gt 0 soclose fdr exit 1 break handler break handler control break or control c e static int break count 0 if break_count 1 first time just try to stop current network operation abort op 1 signal SIGINT break handler reset trap return else second tim
13. structured alphanumeric data On the other hand MDBMS requires both the management of alphanumeric and multimedia data together Multimedia or media data have been introduced as text graphics images sound and signals They all have in common in that a single value or object of that type tends to be rather long i e in the range of 100K to 10M bytes They are often referred to as unformatted unstructured data meaning that they consist of a large and varying number of small items like characters pixels lines or frequency indicators stored together in some way to form a unit They all carry a more complex structure which varies significantly from value to value and is often not known to the user when the object is stored Detecting the type of media which has been stored requires some level of understanding and recognition More detail information about the characteristics and storage requirement of media data can be found MLW88 Due to high storage requirement of media data we are only able to keep them in separate files with the current available technology They can be a separate text a separate image or sound which we call a media object throughout this thesis but actually each of them is only a value of an instance 1n multimedia data In multimedia data a sound for example is an object but it is also the value of the attribute voice The principal task of the DBMSs is storage and retrieval but not processing the
14. 8 digit sound file name consisting of 1 digit each for year month day and hour by using SND NAME C external routine and 2 digits each for minute and second and adds snd suffix to each file name The ERROR store snd hdr function stores the header information of the sound record into a designated file followed by the sound data composed of bit strings In other words it creates the sound object file The ERROR store ssun hdr function stores a unique sound file name and the header information into a designated user defined file as a text file to be sent to the SUN workstations through Ethernet to be used in the MDBMS The FileSize function determines the bit size of the recorded sound data file Ssantex play function is a modification of Antex VP620E play routine used to play a given sound file with a user defined file name This function uses the Read sfilename function to read the unique sound file name from the given sound text file and the Read snd hdr function to read the sound header information from the actual sound object file to play the record and display the header data PLAY C plays a given sound file in unique file name format xxxxxxxx snd and uses SSantex play and Read snd hdr functions discussed above RECEIVER C is a modification of a receiver socket application program in the Excelan The LAN WorkPlace Network Software for PC DOS Socket Library Application Program Interface manual We defined the number 200
15. BUILT DISPLACEMENT CAPT ID PICTURE First we run the MDBMS system and get the main menu on the screen and select the create table option and then we will interactively enter the data in the following order the italics represent the user s responses Enter table name Maximum 12 characters SHIP cr 18 Enter attribute name Maximum 12 characters S NAME cr Select data type of attribute Select 1 integer 2 float 3 c20 4 image 5 sound Select your choice 3 cr Data type c20 20 characters y n cr More attribute in the table y n cr Enter attribute name Maximum 12 characters TYPE cr Select data type of attribute Select 1 integer 2 float 3 c20 4 image 5 sound Select your choice 3 cr Data type c20 20 characters y n y cr More attribute in the table cr Enter attribute name Maximum 12 characters ID cr Select data type of attribute Select 1 integer 2 float 3 c20 4 image 5 sound Select your choice 2 cr Data type integer cr More attribute in the table cr 19 Enter attribute name Maximum 12 characters PICTURE cr Select data type of attribute Select 1 integer 2 float 3 c20 4 image 5 sound Select your choice 4 lt cr gt Data type image cr More attribute in the table y n
16. C has the structure that represents the sound object whose attributes size sampling rate encoding duration and resolution will be stored as a header file prefix to each recorded sound file SND ERRS C module contains a list of possible I O error responses one mig t need during the processing of the programs SND NAME C creates the first four digits of the system created unique sound file name by using the standard time parameters of GMT Greenwich Mean Time First digit stands for year second one for month third one for day and the last digit for hour The last four digits are created by SND REC C program by using minute and second SND_REC C records the sounds and plays them if the user wants to It creates two files one with a unique file name consisting of the sound header information and the sound data and the second one with a user defined name consisting of the unique file name and header information in a text format There are two main functions in this program SSantex record and Ssantex play SSantex record is the routine which records a sound file from a standard connection and creates a sound object file with a unique name and a text file with a user defned name This function is mainly a modified version of Antex VP620E record routine and uses the following functions Generate filename ERROR store snd hdr ERROR store ssun hdr and long FileSize The Generate filename function produces 52 a unique
17. Figure 6 MDBMS Table Array and Table List catalog tables The MDBMS catalog is composed of three main tables or arrays Table Array Table List and Att Array tables as shown in Figures 6 and 7 The Table Array contains the attributes table name table key att count and att entry where table name denotes each different relational table table key denotes the number that will be assigned to the media in the create table operation att count denotes the number of attributes in the relational table and att entry gives the starting point of the table in the Att Array table 27 The second one the Table List array contains the integer numbers which point to the entries in Table Array and it is created for database maintenance purposes The last table the Att_Array is composed of att_name data_type media_id next_index and value_entry attributes where att_name denotes the name of the attributes in the created relations data_type denotes the data type for each attribute including the formatted and media data media_id column is set to 1 for media data types and 1 for non media attributes next_index is the pointer to the next attribute in a given relation and value_entry is the pointer to the Value_Array arrays Figure 7 MDBMS Att Array catalog table The Value Array arrays consist of five arrays and contain the attribute values for each attribute entry in the Att_Array table Figure 8 There are
18. This program helps to promote better understanding of the sound management integration However these routines will not be invoked during the normal operations of the MDBMS prototype They are used strictly for testing and debugging purposes When this happens the user runs the sdemo program in the SUN workstation The system in turn asks for the name of the remote PC which has the Antex VP620E digital audio processor board for the current system they are pcluml and pclum2 This is the PC that will be used for sound management including the playing of the sound files remotely Figure 12 shows the main menu after the PC has been selected To play the sound files stored in the PC equipped with the sound card we use the socket abstract model that we introduced in the previous section First the PC end of the 47 SUN REMOTE SOUND PLAY MANAGEMENT 1 Play Sound File in PC 2 Display Sound File Header Information 0 Exit Enter your choice numbet Figure 12 The main menu of the SUN Sound Management system has to be ready to receive the play commands from the SUN workstation before the desired sound file can be played We have a receive bat file in the PC which serves to receive the play commands and then play the sound files This routine is embedded in a loop to allow the PC to continuously receive and play the sound files as desired by the user On the other end of the system in the SUN environment we have the main menu i
19. ake ake ooo oe ak ake ae ade ade ake ake ak ake ake ake ae e ade ake ake ake ake 7 SSantex_play filename char filename primary input file declarations int port useint int vpfunction samplerate int state error sec hundsec overload int monitor 1 record monitor always on int srate ERROR struct SND HDR hdr Executable Statements 81 if read snd hdr filename amp hdr 0 read header values to set parameters printf WRCURRENT FILE s n filename print NnNSIZE 1 bytes hdr s size printf NSAMPLERATE d Hz n hdr s_samplrate printf NENCODING d 0 none 1 ADPCM n hdr s_encoding printf NDURATION 5f secw hdr s duration printf NRESOLUTION d bits samplenm hdr s resolution vpfunction BEGIN alert to the driver port 0x280 useint 2 VP620 amp vpfunction amp useint amp port if hdr s samplrate 8000 srate 0 else srate 1 vpfunction PLAY VP620 amp vpfunction amp srate filename open file vpfunction STATUS printf nPlease press lt lt SPACE BAR gt gt to stop playing NN do VP620 amp vpfunction amp overload amp hundsec amp sec amp error amp state printf State d Error d Sec d d Overload d r state error sec hundsec overload while kbhit amp state 3 these statements always required to close t
20. and the previous value of digitized amplitude instead of the digitized amplitude for each sample An improvement to differential PCM is achieved by ADPCM encoding technique which predicts the next value by taking a few of the previous samples and extrapolating them Sa88 The rest of the registration data is related to the storing of an encoded sound signal in the computer The resolution is the number of bits used for each sample of sound signal the size denotes the number of bits used to store the sound media and the duration is the number of seconds to play the sound file and is based on the sampling rate and the rate of the playback device 36 The discussion presented above is summarized from the predecessor M S Thesis by Sawyer Sa88 and more detail information about general sound characteristics and techniques can be found there B SOUND MEDIA OBJECT The sound media object as shown in Figure 2 is composed of three parts registration data raw data and description data The registration data and the raw data parts are created in the PC environment and the last part is created in the MDBMS during tuple insertion as stated earlier When we record a sound a file consisting of the registration data like file size sample rate encoding technique duration and resolution and the raw data which is the bit string of the encoded digitized sound is automatically created by the sound management application program A unique fi
21. as the most appropriate model for the task LM88 It was proposed that media data types like image sound text graphics and signal be defined and their processing operations constructed MDBMS User Interface Figure 1 The Architecture of the MDBMS The new system will be able to support the operations through this structure processing the formatted and multimedia data For example the system can now process data in the relation EMPLOYEE name age salary photo where name age and salary are the formatted data types and photo is the image media data type The detail operations on the image data type is given in LM88 Raw media data like a pixel matrix of an image or the bit string representation of a sound or signal are obtained from the process of digitizing the original media object Processing the raw data needs information like resolution pixel depth colormap sampling rate etc that describe the techniques used to capture encode the media data This textual data part was named as Registration Data In addition to registration data another textual description named as Description Data has been attached to the raw data for the purpose of performing content search because automatic recognition of the media data by computers is beyond the state of the art today LM89 Description data strong voice talk fast Figure 2 Structure of a Media Object SOUND This media object model essen
22. displayerr READ read error return SOUND ERROR 79 if fclose f 0 displayerr WCLOSE close error retum SOUND ERROR return ERROR FREE RRR ee hob bebo eoe eoe READ SOUND HEADER ak ake ke ke ale afe ak oe oc ake a ake oc oe ake akk oc ak jk oe afe ake ke This function reads a header from a designated file and returns the header to the caller with the variousfields updated ae 3e NOM coke ke e ke ke o deo oe oco RR ae 34e afe ahe afe ake A FIO ake akk ak AK Read_snd_hdr filename h char filename output struct SND HDR h sound object record FILE f int num 1 only one header if f fopen filename rb NULL open for reading displayerr ROPEN ann_play no_file return SOUND ERROR read the header from the predesignated input file if fread h sizeof struct SND HDR 1 f num displayerr READ read error ann play not read return SOUND ERROR if fclose f 0 displayerrrWCLOSE close error return SOUND ERROR return ERROR FREE ak kk kk kk ahe sie ake ake ale ak ake ake ake k ANTEX PLAY FUNCTION This is the actual function that plays the sound Its input is a filename A successful play will return a 0 to the caller Failure will return an error message MOOR o eoo ake ke ae ahe ake ahe
23. file the Antex Play function opens this file reads the file name and plays the sound file After that the PC is again ready to receive another sound file name from the send socket in SUN through the Ethernet 49 In the following chapter we will present the implementation detail of the sound management interface which we discussed throughout this chapter 50 V IMPLEMENTATION OF SOUND MANAGEMENT INTEGRATION In this chapter we discuss the implementation details of the MDBMS sound management integration one can find the program code related to this implementation discussion in the appendices of this thesis First we will present the implementation details of the sound management in PC and then the sound management integration to the MDBMS prototype A SOUND MANAGEMENT IN PC The main issues in the PC sound management are recording sound and receiving the remote play commands from the MDBMS through Ethernet and playing the desired sound files We use the following programs written in C language to manage the sound data in the PC SND REC C sound record SND STRU C sound structure SND ERRS C sound errors SND NAME C sound file name PLAY C sound play RECEIVER C receiver socket RPLAY C remote sound play 51 The main program for sound recording is SND_REC C and this program uses the modules SND STRU C SND ERRS C and SND_NAME C besides the other library functions SND STRU
24. files when requested by a remote host The filename which is sent by the remote host SUN first is stored into a file and then this program reads that file playfile and fetches the sound file name and plays it MAM ae je je ee ehe he hehe kk je e ehe kk AH ahe ale 34e ake abe ake 3k ake ae ale ake AAA A ale ahe ake ake 34e ake ake ahe ae ake ake ake ake ake ake ake ake HH include lt stdio h gt include lt sys types h gt include lt time h gt include snd_stru c include snd errs c define ERROR FREE 0 define SOUND ERROR 1 define BEGIN 1 ANTEX VP620E commands define SETREC 2 define START 4 define STOP 5 define STATUS 6 define PLAY 8 define END 9 char no file 08hh0053 snd previously recorded announce files char not read 08hh0214 snd The annouces in these files are There is no such a file in memory The given file can not be read ee afe afe be be be afe afe oe ake ake ke PLAY ANNOUNCE FILES x ok 4 ak ak ak ak ok ak ake ae ok ake ake ake 3k sk This function plays the previously recorded warning announce files I Announce play filename 77 char filename sound filename declarations int port useint int vpfunction samplerate int state error sec hundsec overload int monitor 1 record monitor always on int srate ERROR struct SND HDR hdr Executable Statemen
25. found in the thesis by Pongsuwan Po90 In this thesis we will concentrate on the storage and management of sound data using an IBM compatible personal computer connected to the main database system through a local area network In the following chapters first we will concentrate on the MDBMS system and then talk in detail about sound management integration In Chapter II we will discuss the following subjects the development of the MDBMS project before this thesis including the architecture of the system the system environment in which the MDBMS is built and the software and hardware requirements In Chapter III we will discuss the high level user interface design the MDBMS catalog design the processing of structured data in INGRES and the processing of multimedia data in the extended sysiem which currently includes sound and image and the parser that is used to process natural language descriptions of the media data Chapter IV concentrates on an overview of sound characteristics and the sound management in the PC the details of hardware and software requirements for sound management and u 2 sound management user interface design In Chapter V we will talk about the implementation details of sound management in both the IBM compatible PC and the SUN environments and the PC SUN linkage Chapter VI will summarize and present the conclusions The program codes for the sound management in PC and the codes for the sound management in
26. gt include snd stru c include snd errs c define ERROR FREE 0 define SOUND ERROR 1 define BEGIN 1 ANTEX VP620E commands define SETREC 2 define START 4 define STOP 5 define STATUS 6 define PLAY 8 define END 9 int VP620 Read Sound Header This function reads a header from a designated file and returns i the header info to the caller with the variable fields updated Lo TEETH E Read_snd_hdr fname h char fname given sound file struct SND HDR h sound object header FILE f int num 1 only one header if f fopen fname rb NULL open for reading displayerr ROPEN open file error return SOUND_ERROR read the header info from the predesignated input file if fread h sizeof struct SND_HDR 1 f lt num displayerr READ read error return SOUND_ERROR if fclose f 0 displayerrrWCLOSE close file error return SOUND ERROR return ERROR FREE EERE EERE EEE ER EEE ANTEX PLAY FUNCTION ake ale xke oj ke ae oe ake ake akc ak afe ake ok ak oe afe ak ae ake ak afc afc ak ak akk ak This function plays the given sound file SSantex play filename char filename sound filename declarations int port useint int vpfunction samplerate
27. gt to stop State 2 Error 0 Seconds 5 65 Overload 0 lt lt SPACE BAR gt gt End of recording session Registration Data Values of the Recorded Sound File FILENAME 08dh5255 snd SIZE 21584 bytes SAMPLERATE 8000 Hz ENCODING 1 0 none 1 ADPCM DURATION 5 65 sec RESOLUTION 8 bits sample Please press ENTER to continue lt cr gt The next screen displays the main menu again and the user can record another sound or play the recorded sound files or quit the program The status information like State error seconds and overload which are displayed on the screen during the sound recording are defined by the Antex record function and can be found in the Antex VP620E user manual For example the parameter options for state status are O for idle 1 for playing 2 for recording and 3 for waiting to play The attributes of the registration data encoding and resolution are previously set for this prototype and they can not be changed by the user If the user wants to play the previously recorded sound file he selects the play sound option 2 lt cr gt PLAY SOUND Enter the file name to play user defined yavuz cr CURRENT FILE 08dh5255 snd SIZE 21584 bytes SAMPLERATE 8000 Hz ENCODING 1 0 none 1 ADPCM DURATION 5 65 sec RESOLUTION 8bits sample Press lt lt SPACE BAR gt gt to stop playing State 1 Error 0 Sec 5 65 Overload 0 END OF PLAY Plea
28. hehe ee ae ae be ke EERE SND REC C Sk de ke ake be be de he be e oe e ake be de ake ak akk je o ke kk oe ale ke Title SOUND RECORD AND MANAGEMENT IN PC Author Yavuz Vural ATILA Rank LTJG Turkish Navy Advisor Prof Vincent Y LUM Date 27 August 1990 Revised 18 September 1990 Description This program is designed for ANTEX VP620E sound processing card and it records a sound data file from a standard input connection i e microphone tape recorder It creates an header file automatically which includes file size Bytes sampling rate Hz encoding technique duration Sec and resolution Bits sample And also it plays the sound file which is recorded previously Ac afe afe de je je e e je afe afe be je je oe obe be je ae je afe KRR amp amp include lt stdio h gt include lt sys types h gt include lt time h gt include snd stru c include snd errs c include snd name c define NAME LENGTH 13 define ERROR FREE 0 define SOUND ERRCR 1 ftdefine BEGIN 1 ANTEX VP620E commands define SETREC 2 define START 4 define STOP 5 define STATUS 6 define PLAY 8 define END 9 int VP620 5 Generate A New File Name Produce a unique 8 digit filename for the recording composed of 1 digit each for year month day amp hour and 2 digits each for minute and second Each sound
29. if fopen stfilename r NULL open for reading displayerr ROPEN return SOUND ERROR read the header from the predesignated input file fscanf f 96s sfname fclose f return moe eee ke bebe eek ke koe do ANTEX PLAY FUNCTION ak ak oe obe abe be ake oe oe o oe be ok ak ok oe oje oj oc ak ok ake ake kc ake ok ak This function plays the given sound file SSantex play stfilename char stfilename given text sound file int port useint declarations int vpfunction samplerate int state error sec hundsec overload int srate char filename 13 struct SND HDR hdr int monitor 1 record monitor always on ERROR Executable Statements Read sfilename stfilename filename if read snd hdr filename amp hdr 0 display header values for information 73 printf nCURRENT FILE s n filename printf n tSIZE ld bytes n hdr s_size print ASAMPLERATE d Hz n hdr s_samplrate T print AENCODING 96d 0 1 ADPCM n hdr s_encoding printf NADURATION 5 secn hdr s duration printf NRESOLUTION 46d _bits sample n n hdr s_resolution vpfunction BEGIN alert to the driver port 0x280 useint 2 VP620 amp vpfunction amp useint amp port if hdr s_samplrate 8000 srate 0 else srate 1 vpfunction PLAY VP620
30. object can be identified by the snd suffix DR te Ps SUD DENS WESTERN Generate filename sound filename char sound filename unique sound file name char p struc tm t time t current time current time time NULL t rmtime amp current time sprintf sound_filename 1c 1c 1c 1c 2d 2d s YR t gt tm_year MN t gt tm_mon DAY t gt tm_mday HR t gt tm_hour t tm min t gt tm_sec snd sound filename NAME LENGTH 1 0 for p sound_filename p p f p p 05 return ERROR FREE MM Store Sound Header and Data This function stores a header info into a designated file then reads the recorded sound file buffers the data then writes the buffer into the designated file following the header ERROR store snd hdr fname r temp file char fname given unique sound filename struct SND HDR sound object record char temp file FILE f fg char buf 500 input out buffer int num if f fopen fname wb NULL open for writing return WOPEN if fg fopen temp file rb NULL open for reading return ROPEN num 1 only one header write the header into the predesignated output file if fwrite amp r sizeof struct SND HDR 1 f lt num return WRITE write error while feof fg if fread bu
31. on retrieval process and in this thesis we will talk about the Sound Management Integration of this prototype The detail explanations about the design and implementation of this new prototype will be presented in the following sections of this chapter B SYSTEM ENVIRONMENT Due to various resource constraints the decision in 1988 was to construct a MDBMS on an existing database management system INGRES DBMS was chosen for this purpose and the SUN workstations and servers with the UNIX operating system were chosen to be the system to construct the MDBMS in Because the SUN workstations did not support the sound management at that time an IBM compatible PC was used to store and manage sound data A parser was built for the content search purpose The data connection between the SUN workstation and the other stations like UNIX servers parser and IBM PC is established by using Ethernet a local area network The general layout of the system is shown in Figure 3 A number of restrictions are the consequence of using INGRES First the INGRES version in which the original MDBMS prototype was constructed does not support user defined abstract data types Second INGRES allows maximum 500 characters to be stored for a given attribute Third it does not allow its user to get the catalog information 11 readily Fourth an intermediate interface below the SQL language is not available in INGRES The restrictions mentioned above affected
32. return 6 break return 7 break return 8 break return 9 break return a break return b break return c break return d break return e break return f break return g break return h break return i break return j break return k break return l break return m break return n break return o break 92 RECEIVER C ak ae be de e oe be e je ae be de ale oe je be abe ake oe oj ode ahe je ae afe de afe afe ae afe kk This program code is a modified version of a receiver socket application program in the Excelan The LAN WorkPlace Network Software for PC DOS Socket Library Application Program Interface manual This program receives th sound file name sent bv the sending socket in the MDBMS and stores it into a file include lt sys extypes h gt include lt stdio h gt include lt ctype h gt include sys exerrno h include lt sys socket h gt include lt netinet in h gt include lt fcntl h gt include lt signal h gt ifdef LATTICE include lt error h gt else include lt ermo h gt ftinclude lt sys types h gt include lt sys stat h gt endif AF INET struct sockaddr in recv socket AF INET struct sockaddr in send socket ifdef LATTICE define O BINARY O RAW Hendif defi
33. the attributes and their data types of the relation OFFICER The attributes and data types are entered into the Att Airay table starting from the first available row in this table and the corresponding index number of the first attribute is stored as att entry attribute in the Table Array table For our example the first attribute o id and its data type are inserted to the eighth row in the Att Array table Figure 7 and the index number eight is transferred to the Table Array table as att entry Figure 6 The remaining five attributes and data types are inserted to the Att Array table one by one The next att entry value for a third relation is 14 In the Att Array table the next index attribute gives the index number of the next row For implementation purposes the next index value of the last attribute of each created relation is an end mark 1 instead of next row number After the completion of the relational table creation the user can modify the relation by changing the table name attribute names and data types or insert a new attribute into the table or delete an attribute from the table as we said earlier 30 The system catalog discussed above will be used by all the operations in the MDBMS The use of array index compared to the use of pointer linked list structure is judged to be superior it saves a lot of time in searching the catalog tables and simplifies the implementation as well However while attributes are r
34. those the MINOS project at University of Waterloo is an object oriented O O multimedia information system that provides integrated facilities for creating and managing complex multimedia objects Ch86 an O O database management system named ORION has been developed at MCC in Austin Texas which contains a Multimedia Information Manager MIM for processing multimedia data WK87 the IBM Tokyo Research Laboratory has developed two mixed object database systems which are named as MODESI and MODES2 KKS87 and in Europe there is ESPRIT project designing a multimedia filing system called MULTOS Be86 BRG88 A discussion of these projects is presented MLWS88 LM89 will not be repeated here Recently multimedia management in the personal computers becomes available by using hypertext and hypermedia The concept of hypertext is very old it has been transferred to computer systems since 1960 s Originally intended to manage arbitrarily linked text segments it has been extended to manage images and sound and has become Hypermedia MLWS88 The hypertext and hypermedia data management in the Macintosh computer with a hypercard application has many users including the ARGOS project being developed at Naval Postgraduate School WNTA89 The hypertext and hypermedia data management uses the hierarchical data structure approach in which users cannot query the data as done in the conventional DBMSs but have to follow
35. 0 as a virtual port between this socket and the sending socket in the SUN workstation end There are two options for this socket to display the received data on the screen or to store the data into a designated file We have chosen the second one and now the receiver socket writes the 53 sound file name sent by the MDBMS through Ethernet into a file named playfile We add this filename as an argument to store the sound file name when the receiver socket program receiver file playfile is called RPLAY C plays the previously recorded sound files when they are requested by the MDBMS This program first opens a designated file which is given as an argument for our case rplay playfile then fetches the sound file name written by the receiver socket and plays the sound record The functions in this program are SSantex play Read snd hdr Read filename and Announce play We have talked about the first two functions already the Read filename function which opens the given file playfile and reads the sound file name and the Announce play function which is used to announce a message to warn the user in case of an error There are two batch files in the PC sound management system other than the programs discussed above RECORD BAT and RECEIVE BAT The RECORD BAT file contains the following commands play 084m4325 snd snd rec play 084m4434 snd When the user enters record command the system executes these comm
36. Aceession Por NTIS GRAaI DTIC TAB Unarncunce Justification NEN Dtstcibuitenz Avallab itp Je ees E iii TABLE OF CONTENTS T INTRODUCTION 1442 hh MURS ACC E PRESSURE ES 1 BACKGROUND EE MEME eta YR RR E RS 1 B AN OVERVIEW 2 ga Xa Va aee ee n AE S 4 IL SURVEY OF THE MDBMS Rd ES ESSE Noe 6 A DEVELOPMENT OF MDBMS 6 B SYSTEM ENVIRONMENT e IS Te we 11 III INTERFACE AND CATALOG DESIGN OF THE MDBMS 14 A USER INTERFACE DESIGN 14 l The Main 14 2 MDBMS Applications 18 B CATALOG DESIGN 5 24528 du wr pq t 27 SOUND MANAGEMENT i hehe e Arr eei sdb iw 35 A AN OVERVIEW OF SOUND CHARACTERISTICS 35 B SOUND MEDIA OBJECT 37 iv C HARDWARE AND SOFTWARE REQUIREMENTS 38 1 Sound Management Requirements in IBM Compatible 38 2 The PC SUN Linkage Requirements 41 3 The MDBMS Sound Management Integration Requirements 43 D SOUND MANAGEMENT USER INTERFACE DESIGN 43 V IMPLEMENTATION OF SOUND MANAGEMENT INTEGRATION 51 A SOUND MANAGEMENT IN PC
37. ERATE d Han hdr s samplrate printfNENCODING d 0 none 1 ADPCM W hdr s encoding print NDURATION 5f secn hdr s duration printf NARESOLU TION d bits samplevW hdr s resolution store header and data into designated file if err store snd hdr filename hdr newname OK displayerr err if err store ssun hdr sfilename hdr filename OK displayerr err print nNNPlease press ENTER to continue 71 MM Read Sound Header This function reads a header from a designated file and returns the header info to the caller with the variable fields updated X a P Read snd hdr fname h char fname given sound file struct SND HDR h sound object header FILE f int 1 only one header if f fopen fname rb NULL open for reading displayerr ROPEN open file error return SOUND_ERROR read the header info from the predesignated input file if fread h sizeof struct SND_HDR 1 f lt num displayerr READ read error return SOUND ERROR if fclose f 0 displayerrrWCLOSE close file error return SOUND ERROR retum ERROR_FREE Read_sfilename stfilename sfname char stfilename given text sound file name char sfname actual sound file name FILE f 72
38. INGRES DBMS to manage the standard data for the multimedia data we need to establish our own database structure to handle the media data A main issue on processing the multimedia data is how to address the contents of the media data Handling the content search issue is not possible with the conventional methods used in the current DBMSs so we need to use an abstract data type concept to let us handle the media data And also we need to have some parameters together with the raw media data to display the media data e g size pixel depth resolution encoding and colormap for the image data and size sampling rate encoding duration and resolution for sound data The media data used to implement abstract data type concept is composed of the following parts registration data raw data and description data T e 58 registration data part includes the parameters for the interpretation or display of the media data has a fixed format because the formats or field lengths of the parameters are known The raw daia is in a bit string format like a pixel matrix of an image or the bit string representation of a sound record And the description data that describes the object content of the raw data and is used for content search purposes is entered by the users in natural language form A parser was constructed for processing natural language description data A dictionary or lexicon is prepared for the MDBMS and the user is restricted to use
39. aduate School Department of Computer Sciawce Monterey California September 1990 Sawyer G R Managing Sound in a Relational Multimedia Database System Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California December 1988 Stewart R Design and Implementation of a Multimedia DBMS Modification and Deletion Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California in preparation Tanenbaum A S Computer Networks Prentice Hall Inc 2nd Edition 1988 63 Th88 WNT89 WK87 Thomas C A A Program Interface Prototype for a Multimedia Database Incorporating Images Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California December 1988 Wu C T Nardi P Turner H Antonopoulos D ARGOS Next Generation Shipboard Information Management System Report no NPS 52 90 006 Naval Postgraduate School Department of Computer Science Monterey California December 1989 Woelk D and Kim W Multimedia Management in an Object Oriented Database System Proc 13th International Conference on VLDB Brington England September 1987 APPENDIX A PC SOUND MANAGEMENT PROGRAM CODE The following program code is either created or modified from the code written by Sawyer Sa88 and Antex VP620ESE driver software for the implementation of sound management in the PC ehh ek hehe be ee ek he
40. agei image facts to be used by Prolog Po90 When the temporary tables are obtained after processing 33 each condition stated by the user a join operation is done by the system to display the result table to the user i The detail information about the catalog management table creation and tuple insertion related to catalog tables can be found in Pe90 and the retrieval operation in Po90 IV SOUND MANAGEMENT A AN OVERVIEW OF SOUND CHARACTERISTICS In this chapter some of the basic characteristics of sound which are used in the sound media data type are discussed As we stated earlier the sound media data type includes sampling rate encoding technique resolution size and duration as registration data and these data are used to play the sound data in the system Sound can be defined as vibrations in a waveform in a conducting medium like air water wire etc The sound we hear in the nature is in the analog waveform which is a continuous wave On the other hand we store the sound in the computer in the digital form as a sequence of bits as ones and zeros The conversion of the sound in the analog form to digital form is accomplished by using a technique called sampling Under certain conditions an analog sound signal can be completely represented by knowledge of its instantaneous values or samples equally spaced in time This sampling process is done by using an analog to digital converter ADC The con
41. ailable and needed Personnel Database We can store the information about the personnel like name rank job age weight height salary marital status etc with the photographs fingerprints and voice prints of the personnel News Archive Database We can store the pictures and the narrative explanation of an event or accident or a ceremony etc C FUTURE WORKING AREAS Currently not all the database operations of the MDBMS prototype completed More work needs to be done for the modification and deletion operations in the databases the enhancement of the querying capability like nested queries and more effective help utilities The system can be improved with a more user friendly interface similar to the modern graphical interfaces like using mouse clicks in colorful windows instead of typing after each prompt At this time three more M S thesis students Aygun Peabody and Stewart Ay91 Pe91 St91 are working on these areas 61 Ay91 Be86 BRG88 Ch86 Du90 EN89 KKS87 LMS88 LM89 LIST OF REFERENCES Aygun H Design and Implementation of a Multimedia DBMS Complex Query Processing Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California in preparation Bertino E Gibbs S Rabitti F Thanos C and Tsichritzis D A Multimedia Document Server in Proc 6th Japanese Advanced Database Symposium Tokyo August 1986 Information Proce
42. ands in order The play commands play the previously recorded interface announce files like Welcome to the sound management system and Thank you for using sound management system Have a nice day And between these announcements we have SND REC C program record and play sound as discussed above The RECEIVE BAT file contains the following commands 54 receive erase playfile receiver file playfile rplay playfile erase playfile goto receive When the user enters receive command the system executes these commands in order and after the execution of the last command goes back to the first command and waits in a loop for the next action There are two erase playfile commands because the receiver socket can not write into a nonempty file In the case of an existence of an unerased playfile file at the beginning of the loop we put the first erase playfile command just before the receiver socket command The system waits for the sound file name to be sent from the MDBMS end through Ethernet after clearing the playfile When a sound file name is sent from the MDBMS the receiver socket receives the name and writes it into a file playfile Then with the next command rplay playfile the system opens this file and fetches the sound file name and plays the sound record Finally the system erases the playfile and waits to receive and play the next sound file name These are all the programs and routines that we use for the PC so
43. c ake ok ake ke sk ake ak afe abe abe be ake jc abe afe ak ok oc ake ie ak oe This function records a sound file from a standard connection and creates two output files one sound object file and the other one text file contains the registration data and the unique sound file name xkckclolokekekcokelololokokekolekelololokolelokookelelelokaolelelelolelelelololelelelelelelelelelolelelelolelelelekerolelelekelekejoleleleielek Ssantex record filename sfilename char filename sound object file name char sfilename sound text file name int port useint int vpfunction samplerate int state error sec hundsec overload int monitor 1 record monitor always on long int sz int srate sencode sresol float sdur char c int pi storage allocation stmts unused i 0 char newname ERROR err struct SND HDR hdr Executable statements generate filename filename puts Please ENTER a filename to transfer the sound text file to SUN 69 gets sfilename printf Please ENTER 0 for sampling rate 8 Khz n printf 1 for sampling rate 16 Khzw scanf 76d amp samplerate c getchar vpfunction BEGIN vpbegin port 0x280 use default IO address useint 2 use interrupt 2 VP620 amp vpfunction amp useint amp port wake up call to driver vpfunction SETREC vpsetrec sencode 1 ANTEX recording w 8bit resolution
44. cket routines to the application programs which are already written in C language When a communication is desired between both ends of our system the application program makes a request for the opening of a socket and all communication takes place through that socket We use the sending and receiving sockets on both the PC and the MDBMS sides of the network The detail information about the implementation of sockets will be presented in the next chapter We can list the necessary hardware and software for the PC SUN linkage as following 42 Excelan EXOS 205T Intelligent Ethernet Controller board Excelan LAN Local Area Network Workplace Network Software for PC DOS TCP IP Transport System Excelan LAN Workplace Network Software for PC DOS Socket Library Application Program Interface An Ethernet computer network Socket library routines for SunOS 3 The MDBMS Sound Management Integration Requirements On the MDBMS side we do not need that much hardware and software because the system is already established in the SunOS environment by using UNIX C After the establishment of PC SUN socket connection all we need to do is to include the necessary socket routines in the MDBMS prototype and a couple more routines into both the PC and the SUN systems to handle sound files and create the required sound media relations D SOUND MANAGEMENT USER INTERFACE DESIGN The user interface that we talk about in this
45. defined file name The description part of the sound media object file is created in the SUN workstation environment during the tuple insertion process Only the second file sound text file is transferred from the PC to the MDBMS environment to be stored in the sound media relations When the user wants to play sound the system only sends the unique sound file name to the PC through the local area network Ethernet There are two application systems working simultaneously one in the MDBMS for sending a sound file name and the second one in the PC waiting to receive the sound file name sent from the MDBMS for playing We used the socket abstract model through Ethernet to accomplish this connection between these two application systems B APPLICATION AREAS We can imagine many application areas for the current MDBMS prototype with image and sound processing capability We can give the following application scenarios as an example Electronic Warfare Training We can store the parameters of the electronic devices like radars the picture of the intercepted signals and the sound record of these signals The EW operator can guess the name of the radar and parameters by looking at the picture of the signal and listening the sound record Ship and Weapon Database We can store the formatted information about the ships and weapons like name type length number displacement power range etc and the pictures and sound record if it is av
46. des some basic routines referred to as commands which can be called from high level languages like Basic Pascal and C Once the VP620ESE driver software is activated these routines will run in the background independent of the status of an application program 2 The PC SUN Linkage Requirements The task of connecting the PC to MDBMS in the SUN environment for the purpose of remotely interacting with the PC as a backend server from the MDBMS was the main and critical issue in the beginning of this thesis work First we tried to establish a closed net by using a null modem between the SUN workstation and the PC but the software PROCOMM we had for this connection did not fulfill our specifications because this connection does not allow the PC and the SUN workstations to send and receive commands between them and this connection requires to have the DOS in both ends More over it gives us difficulties of connecting multiple PC s to multiple SUN workstations Then we concentrated on using the Ethernet computer network which is available and has already been used to connect the SUN workstations The IBM compatible PC connection to the Ethernet was accomplished by using Excelan The LAN Workplace network software The main communication between the SUN and PC was to play a given sound file name remotely in the PC Consequently we decided to use remote shell RSH command which is available in both the SUN and PC network softwa
47. digital format for PC disk storage and when one needs to playback it is possible to convert the digital stored data back to the analog signal for playing with a speaker These processes correspond to encoding and decoding of the sound signal as we said earlier The VP620E samples the audio waveform at either 8 16 Khz using ADPCM encoding technique The input sound signals can be captured in the system either by using a microphone together with an amplifier because the input signals must be at least 1 volt RMS in order to be accepted by the sound board as being above the threshold noise level or the output port of an audio device like a cassette player radio etc Each sample is converted into an 8 bit digital number which is then compressed encoded using ADPCM by the Digital Signal Processor DSP chip resulting in a 4 bit sound data sample Sa88 The output process is the reverse of the capturing process The 4 bit sound data is decoded back to 8 bit digital samples by the DSP chip and the resulting samples then go to the speaker through amplifier stage It is also possible to play sound in a background mode under DOS interrupt control allowing the user to perform other operations in the MDBMS while a sound record is being played This is accomplished by piping the sound data directly from the database to the DSP chip of the Antex sound board instead of RAM Sa88 The Antex Audio Processor board comes with a software package and inclu
48. e try to clean up then quit errexit user abort P ehe ke ke ke coe kj ke ke kk ke e eee ake je ake je ke ke ke AA khe ke ahe afe afe 3e kk AA AN A A A A AR AA AA AA AA AA HAA A HA HACE APPENDIX B SUN SOUND MANAGEMENT PROGRAM CODE The following programs are written for the setup to play a sound file in a remote PC and to read the text file transferred from PC to the SUN environment that contains the header information of the recorded sound The main program is integrated into the main MDBMS program as modules and one can find the main program code in Pe90 P090 The program code is created or modified from the code written by Sawyer Sa88 and the sending socket routine in the main program is a modified version of the Internet Domain Stream Connection program in the SUN Microsystem Network Programming manual chapter 8 A Socket Based Interprocess Communications Tutorial A Se aaao oak STE MO FERRER ae ake ake ake EHH ake ak ak ake ak ak ake ake ak ake ake ak ake ake 3k ake ak ak ak k Title SUN SOUND INTERFACE Author Yavuz Vural ATILA Rank LTJG Turkish Navy Advisor Prof Vincent Y LUM Date 5 September 1990 Revised 20 October 1990 Description This program is designed to play the previously recorded sound file in a remote PC and to read the sound header information from a text file which is transferred from PC by using the file transfer protocol FTP Neh a ad
49. e he ke be ke e ke e joe je eee e dele ade ae dele ke ke ae de ade de ae ade ade ae e ke ahe ake abe 38e ade ale ake ake ae ake ale afe abe afe ahe ahe ale ade ake abe ae ae ake ahe ake ake ake ake ak ake ake ake ake ake ake ake aak include lt stdio h gt include lt sys types h gt include lt time h gt include lt sys socket h gt include lt netinet in h gt 97 include lt netdb h gt include snd str c define SOUND ERROR 1 CONNECTION TO PC ake ak obe ae kok ak ak ak ok afe akk ak ake ake ok ok ae ale ake aie ak ak ak This part connects SUN to PC and sends the sound filename to play in PC ae ae ake ake ake ake ake 24e dbe ae ake ahe ahe ab ahe abe ake ale ake ahe ake abe ake RIKK SendSocket filename pcname char pcname remote PC host name char filename Short 2000 virtual port number between SUN amp PC int sock struct sockaddr in server struct hostent hp gethostbyname char buf 1024 Create socket sock socket AF INET SOCK STREAM 0 if sock lt 0 perror opening stream socket exit 1 Connect socket using name specified by command line server sin family AF INET hp gethostbyname pcname if hp 0 fprintf stderr 76s unknown host pcname exit 2 bcopy char hp gt h_addr char amp server sin_addr hp gt h length server sin port htons port if connect sock
50. e operations as if they all belong to only one system III INTERFACE AND CATALOG DESIGN OF THE MDBMS The MDBMS prototype user interface design that includes the high level DBMS operations and applications and catalog design that includes the catalog tables and their interrelationships will be discussed in this chapter As previously mentioned in the introduction chapter two more M S thesis students have been involved in the current design of the MDBMS and consequently the subjects in this chapter are also presented in varying depth and breadth in the companion theses Pe90 Po90 A USER INTERFACE DESIGN 1 The Main Operations In this section we will discuss the MDBMS from the user s point of view The allowed operations in the MDBMS are the same high level operations as in a standard DBMS for formatted data namely table creation data insertion retrieval modification and deletion Currently most the first three operations have been implemented and the rest of them are in progress Ay91 Pe91 5191 Because the MDBMS is composed of two different database managers which are INGRES for formatted data and the multimedia manager for media data an extended SOL language structure is needed to specify the queries LM89 But instead of asking a user to enter queries in formal SQL structure the user interface is designed to get these required data interactively in a user friendly way After getting the required data for 14
51. equired to be unique within a user relation same attributes names are permitted in different relations Although this situation works fine for the formatted data because INGRES manages this kind of data and confusion will not arise it creates problems for handling media data In the MDBMS prototype a separate relation referred to as media relation in INGRES is created for each media attribute The name of this media relation is the same as the name of the media attribute To avoid confusion and keep the media relations distinct when the same attribute name is used in more than one user defined relations the names of these media relations are appended by suffixes in this case numbers corresponding to the unique system identifiers assigned to the user relations Po90 When the user wants to create a relation the procedure followed by the system each time might be different depending on the data types of the attributes of the new relation If all the attributes of the new relation are formatted data type then the entered information is transferred to INGRES via the create table command of SQL On the other hand in the existence of media data types the system asks INGRES to create a media relation for each media attribute with the same attribute name in the Att Array table and with a suffix No duplication of attribute names is permitted within the same relation The attributes of the media relation depends on the media type the image media rela
52. es The current MDBMS prototype supports the boolean conditions in disjunctive normal form It uses the AND boolean operator inside each group and the OR boolean operator between group conditions group condition is either a single condition or multiple conditions grouped together Conditions not specified in the disjunctive normal form are not supported in the current MDBMS prototype The reason for preferring the disjunctive normal form is to simplify implementation without sacrificing functionality and usability The condition statement for the formatted data is entered with a preceding operator like gt lt gt gt etc and additionally for the character data types the text data should be in quotation marks following the equal sign operator like For the media attributes the media description must be entered without quotation marks Multiple descriptions must be entered one description at a time followed by a blank line when no more description is to be entered After these interactive inputs the system processes the queries and displays the requested data in a table form However while the values for the formatted attributes in the display are really the desired values the values for the attributes of media data types are only index numbers which points to the tuples in the media relations and cannot be understood by the user They are created and used by the system itself In case of the existence of media data the
53. eturn 9 break case 10 return 0 break f case 11 return A break case 12 return B break December char DAY day int day switch day case 1 retum 1 break case 2 return 2 break case 3 return 3 break case 4 retum 4 break case 5 return 5 break case 6 retum 6 break case 7 return 7 break case 8 return 8 break case 9 return 9 break case 10 return a break case 11 return b break case 12 return c break case 13 return d break case 14 return e break case 15 return f break case 16 return g break case 17 return h break case 18 return i break case 19 return j break case 20 return k break case 21 return l break case 22 return m break case 23 return n break case 24 return o break case 25 return p break case 26 return q break case 27 return r break case 28 return s break case 29 return t break 91 case 30 return u break case 31 char HR hr int hr switch hr case 1 case 2 case 3 case 4 case 5 case 6 case 7 case 8 case 9 case 10 case 11 case 12 case 13 case 14 case 15 case 16 case 17 case 18 case 19 case 20 case 21 case 22 case 23 case 24 return v break retum 1 break return 2 break return 3 break return 4 break return 5 break
54. f 500 1 fg 0 load buffer return READ append data from sound data buffer if fwrite buf 500 1 f lt num write buffer return WRITE if fclose f 0 retum WCLOSE close error if fclose fg 0 return WCLOSE return OK 67 Store SUN Sound Header This function stores a header information into a designated file as a text file to send to the SUN for using in the MDBMS etr PM EM ce OS PM ERROR store ssun hdr fname r sfname char fname given sound file struct SND HDR sound object record FILE f if f fopen fname w NULL open for writing return WOPEN fprintf f s 0 sfname store the header info fprintf f n ld 0 1r s_size into a text file fprintf f n d 0 5_ fprintf f n96dN r s encoding fprintf f n96fN r s duration fprintf f n26dNON r s resolution if fclose f 0 retum WCLOSE close error return OK long FileSize i_file char i_file input file FILE f long int f_size if f fopen i_file rb NULL open file displayerr ROPEN if fseek f 0L 2 0 set position rel to end return EOF 68 f size ftell f if fclose f 0 close file displayerr RCLOSE return f_size ANTEX RECORD FUNCTION a
55. file name to the given PC through a virtual port number 2000 In the case of failure in the connection the system displays an error message otherwise the user can continue the other applications in the MDBMS Snd inforead reads the unique sound file name and header information from a given sound text file which is sent from the PC to the SUN workstation 56 The functions in the SDEMO C program SendSocket and Snd inforead are included in the main program of the MDBMS prototype to send the sound file names to the PC and to store the header information into the sound relational tables during tuple insertion The program code related to the SUN sound management is given in Appendix B 57 VI SUMMARY AND CONCLUSIONS A REVIEW OF THESIS The current MDBMS prototype at the Computer Science Department of the Naval Postgraduate School can let the user to capture store manage retrieve and present the image and sound media data together with standard data like numerics and alphanumerics The system although conceptually using a SQL like query interface to process the operations in the MDBMS demands interactively the necessary data from a user in a user friendly way without any formatted SQL statement In this way the users do not need to have wide and deep background or knowledgeable about the database systems In the MDBMS we need to deal with both the standard data and the multimedia data together We use the
56. he file vpfunction STOP VP620 amp vpfunction vpfunction END VP620 amp vpfunction 82 return ERROR FREE else displayerr READ retum SOUND ERROR BERR EREEEREEEREREREREERER ER EE ES MAIN RAK ake ake ake ake ake ake ak ade ake ake ae ake ake akk ak ake ake ake afe afe ake ake ae ak akk ake ak ak 3k main argc argv receives the filename which sound int argc filename is already written inside char argv by PC receiver socket program char filename 12 read filename filename argv 1 read the filename from external SSantex play filename file end of main ke kk joke ke ae ak ae ake ake ak ake ke oe ak ahe ak ake ak ake eoe ak ake ake ake ake ake ake ake ake ake ese ee ee EERE aoak DJ aka ak ak ak ak a ake ERE EEK RR afe ake ahe AHH AAA AH HK Title PLAY SOUND FILE IN PC Author Yavuz Vural ATILA i Rank LTJG Turkish Navy Advisor Prof Vincent Y LUM Date 27 August 1990 Revised 17 September 1990 Description This program plays a given sound file in xxxxxxxx snd format by using the ANTEX VP620E sound processor board ake abe 24e ahe e ee bebe bebe be heb abe je e je ele ede hehehe ke je ee ke abe AE EH ER ae ake AK EERE ke lee BREE 34e ake ake ak ERK ake ake ak ake include lt stdio h gt include lt sys types h gt include lt time h
57. his structure represents the sound object whose features will be stored in the sound file as a header file prefix to each recorded file The database information will consist only of the unique file identifier and the description data akk ake kc ake hehe de je he he oe oj ahe dede ke ae dede oj je ake ale ae je ok ade ade ee ake ake deje dede ake ake ke abe ake ake ae ake ahe ahe ake ake ae e ake ake afe ake ake ake afe ake ale ake afe ake ake ake ak ake ACH ak struct SND_HDR long int s size number of bytes int s saraplrate 8K or 16K per sec int s encoding 1 float s duration time in sec and hundredths int s resolution bits per sample hdr info 88 EEE EEE EERE ake ake ake ake ake ake SND ERRS C ake ake akk afe aje oe ale de afe abe abe be ae oe ake oe akc ake oe afe afe afe abe afe ale ale aje afe ak ale akc This module contains a list of possible I O error responses This list is truly extensible eee He ake ok ake e ake oboe ok ke ake lebe ake afe ahe ake A KR afe ake HR ale ake ake ake ae ake ake ake akk akk HAA EE typedef enum PARS WOPEN WRITE WCLOSE ROPEN READ RCLOSE SRATE OK ERROR void displayerr e ERROR e switch e case PARS printf Incorrect parameters n return case printf Cannot open file for output return case WRITE printf File write error n return case WCLOSE pri
58. imited DESIGN AND IMPLEMENTATION OF A MULTIMEDIA DBMS SOUND MANAGEMENT INTEGRATION by Yavuz Vural Atila Lieutenant JG Turkish Navy B S Turkish Naval Academy 1984 Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN ENGINEERING SCIENCE from the NAVAL POSTGRADUATE SCHOOL December 1990 Author Approved by David Hsiao Second Reader Robert B McGhee Chairman Department of Computer Science ABSTRACT Today in addition to standard data like numerics and alphanumerics it is possible to capture store manage retrieve and present different media information e g sound image graphics text and signals by using the current modern computer technology The Multimedia Database Management System MDBMS project which started at the Computer Science Department of the Naval Postgraduate School in 1988 studies not only the storing managing and retrieving different media information but also the management of the interrelationships among the data This thesis concentrates on the Sound Management Integration of the MDBMS prototype which includes the storage and management of the sound records in an IBM compatible personal computer and the connection and integration of it to the database system in the SUN environment through a local area network after presenting the general overview of the MDBMS the system environment the user interface and the catalog designs
59. iption NOTE One phrase per line end with an empty line j big aircraft_carrier with many airplanes has many missiles of different kinds cr cr Table Name SHIP Order Attribute Data Type Value 1 S NAME c20 Kitty Hawk 2 TYPE c20 Carrier 3 S NO c20 CV63 4 YR BUILT integer 1961 5 DISPLACEMENT integer 81123 6 CAPT ID integer 100 7 PICTURE image HAS VALUE Media Description Att name Ship picture File_name V90266 221112N Description big aircraft carrier with many airplanes has many missiles of different kinds gt gt Any change before insert Before inserting a tuple the user has an opportunity to change the values for each standard attribute and change the media file name or description The process for the sound media type is the same The system asks the user for the sound file name whether to Play the sound before enter the description y n and then to enter the description The create table and insert tuple operations are presented in more detail in Pe90 22 The last MDBMS operation that we will explain in the user interface environment is retrieve Let s assume that the user wants to retrieve the ships built after 1960 with their names types numbers and pictures For this retrieval the query in extended SOL is SELECT s name type s no picture FROM SHIP WHERE built gt 1960 In the MDBMS system the user enters this query interactive
60. le name is assigned to the created file by using the recording date and time group We do not need to transfer the whole file to the MDBMS environment since there is no sound support in the SUN workstations We only send the registration data and the created unique file name of the sound record to the MDBMS system by using the file transfer protocol FTP in the Ethernet computer network The registration data and the file name will be used in processing the queries During the MDBMS applications when a user wants to listen to the sound record a remote play command with the sound file name is sent to the PC sound management through the network A socket abstract model that will be discussed in the next section 37 is used in the both ends of the network to establish the interface between the application programs and the network C HARDWARE AND SOFTWARE REQUIREMENTS The current MDBMS prototype is based on managing the sound media in an IBM compatible PC environment and accessing the sound media by using the sound interface through Ethernet as stated earlier The work related to the sound management prototype in the PC environment as said before has been done by a M S thesis student Sawyer in 1988 In this thesis the previous work related to sound management has been modified The network connection between the SUN workstation and PC and the Sound Management Interface in the MDBMS prototype has been accomplished The sou
61. ly after selecting the third operation from the main menu which is retrieve Upon completion of entering the query the system creates the SQL statements and processes the query An example of a session for entering a query is shown below Select the table s separate by comma lt gt ship cr Table Ship Select the attribute s separate by comma lt gt s name type s no picture cr 4 ship s name s name c20 ship type type c20 ship s no s no c20 ship picture picture image Any condition y n y cr Group condition yn n cr 23 Enter attribute name yr built cr ship yr built yr built integer Enter the condition 1960 Select ship s name ship type ship s no ship picture From ship Where yr built 1960 Record id 1 s name Kitty Hawk type Carrier s no CV63 photo 1 Press ENTER to continue cr Record no 1 Filename n virgo work mdbms 90266 221112 Number 1 Description big aircraft carrier with many airplanes has many missiles of different kinds gt gt Do you want to see the photo n In the case the result contains more than one tuples when a query has been processed the system asks the user whether to display media data like photo or sound one by one in the order appeared in the resulting table Sometimes to answer a query the system must process more than one relational table using join condition
62. ms the description data in natural language form into a more formal representation namely a set of predicates and literals suitable for Prolog processing These predicates state a fact about the content in the media object Thus the set of all predicates that can be used in the descriptions must be defined in the dictionary When a query is entered for media data retrieval it must be also in natural language form and be parsed by the parser media object is selected as the result of the query if and only if the media object description logically implies the query description The detail information about the parser is presented 0090 The implementation of the MDBMS began in 1988 The subcomponent Image Manager of the Multimedia Manager was initiated by a M S thesis student Thomas Th88 Thomas provided the low level functions for processing image database like storing images in a relational DBMS retrieving registration and raw data for display etc Subsequently another student Po90 implemented the high level retrieval operations using the low level fun ons defined by Thomas Po90 The subcomponent Sound Manager of the Multimedia Manager as shown in Figure 1 was implemented by another M S thesis student Sawyer Sa88 Sawyer provided a similar processing capability for the incorporation of sound media data type as done by Thomas for the image data He used an IBM compatible PC for processing the sound data becau
63. n cr Table Name SHIP Order Attribute Data Type 1 S NAME c20 2 TYPE c20 3 S NO c20 4 YR BUILT integer 5 DISPLACEMENT integer 6 CAPT ID integer 7 PICTURE image Any change before create cr Although the example above has no changes the user could change table name attribute names and data types or insert a new attribute into the table or delete an attribute from the table if the user enters y After creating the relational table now we can insert tuples by choosing the second option insert tuple Enter table name Maximum 12 characters for HELP SHIP cr Table Name SHIP Att Name S NAME 20 Data Type c20 Please Enter lt lt c20 gt gt Value if unknown Kitty Hawk cr Table Name SHIP Att Name TYPE Data Type c20 Please Enter lt lt c20 gt gt Value if unknown Carrier cr Table Name SHIP Att Name S NO Data Type c20 Please Enter lt lt c20 gt gt Value if unknown CV 3 cr e eccvosososotosceceececcece Table Name SHIP Att Name PICTURE Data Type image Please Enter image File Name NOTE Enter the Full Path if unknown Inlvirgolmdbms ras cr Display the image before enter the description cr Figure 5 The image of USS Kitty Hawk Enter the description y cr 21 Please enter the descr
64. n which the user can see the header information of a sound text file or play the sound file in the PC As we stated earlier this SDEMO program is used for test purposes and we can not use the user defined sound file name as explained earlier If the user wants to play a sound file from the SUN end of the system he sees the following interface 2 cr DISPLAY SOUND FILE HEADER INFO Enter the USER DEFINED sound text file name 48 yavuz cr FILENAME 08dh5255 snd SIZE 21584 bytes SAMPLERATE 8000 Hz ENCODING 1 0 1 ADPCM DURATION 5 65 sec RESOLUTION 8 bits sample Please press ENTER to continue cr Now the system displays the main menu again Figure 12 and if the user wants to play the sound file 2 cr PLAY SOUND IN PC Enter the sound file name xxxxxxxx snd to play in PC O8dh 755 snd cr The system invokes the send socket routine with the given sound file name and sends the file name to the PC through the Ethernet In case of any failure in the connection the system displays a warning signal otherwise the user can continue the process in the MDBMS without waiting for the end of playing However to play a second sound file one should wait for the end of playing else the PC is not ready to receive again On the PC end the receiver socket receives the sound file name and writes it into a file According to the order in the receive bat
65. nd media management in the MDBMS can be separated into three main parts according to the hardware and software requirements Sound Management in IBM compatible PC environment PC SUN Linkage Integration of Sound Management to the MDBMS in SUN 1 Sound Management Requirements in IBM Compatible PC As stated earlier the main work related to the sound management in the PC has been initiated by Sawyer in 1988 Sa88 In this thesis we modified and improved his program code for the current MDBMS prototype in a menu driven and user friendly fashion We will talk about the implementation details in the next chapter 38 We need the following hardware ard software to accomplish the sound management in an IBM compatible PC environment as shown in Figure 10 BM compatible PC AT with 20MB internal hard disk 80286 or higher Antex VP620E PC compatible plug in Digital Audio Processor board A speaker with a 1 4 standard jack A microphone with an audio amplifier and a 1 4 standard jack Microsoft C 5 0 with standard libraries Antex VP620ESE driver software MS DOS version 3 0 or higher Figure 10 The Sound Management in IBM compatible PC The Antex Electronic s Model VP620E PC compatible Digital Audio Processor board together with other peripheral devices and software components is the main 39 hardware unit used for sound management This board converts real speech or music sounds to
66. ne FILEOFLAG O_CREAT O EXCL O WRONLY O BINARY define FILEPMODE S IREAD S IWRITE define MAXBUF 8192 extern int errno extern int break enabled extern int abort op int fdr 1 receiver socket descriptor int fdw 1 stdout for displaying message sent int filefd 0 input data file intialize to stdin char buf MAXBUF 0 buffer 93 char filename input file name int bufsize 1024 default buffer transfer size int cnt 0 netcnt 0 number of bytes read sent M short port 2000 port address for the connection int break handler amp main argc argv char argv int an Check that the driver is loaded and enable C handling if loaded errexit driver NOT loaded signal SIGINT break handler break enabled 1 process arguments for an 1 an argc if argv an 0 switch argv an 1 case z set buffer si z e if an lt argc bufsize atoi argv an if bufsize lt 0 Il bufsize gt MAXBUF errexit illegal buffer size fprintf stderr bufsize 96d bufsize else errexit expected buffer size break case f name of file to transfer if an lt argc filename argv an filefd open filename FILEOFLAG FILEPMODE if filefd lt 0 errexit cannot create filename else errexit expected filename break
67. ntaining the data need to be transferred to the MDBMS environment before the insertion process when the system asks the user to enter the full path of the media data file the user can give the address of the media file Subsequently the system creates an image id or sound id index depending on the next available row in the media relation and stores the file id as a whole path and the registration attributes by extracting from the media data file Then the user enters the description data for the media as stated before The retrieval operation is the most complex operation of the MDBMS prototype When the system gets the required data from the user interactively it builds an extended SQL query If the query includes only the formatted data it can be transferred directly to INGRES for processing but for the media data a decomposition needs to be done The system will separate the query into two parts one part related to the formatted data and processed by INGRES and the second part is done by using the information in the media relations In the case of the existence of a media description in the condition part of the query like officer with soft voice the system uses Prolog to search the media data contents in the description attributes of the related media relation Natural language descriptions are handled by means of a parser which transforms the description data into the Proleg predicates and literals to be deposited in a file named im
68. ntf Cannot close output file return case printf Cannot open file for input return case READ printf File read erro return case RCLOSE printf Cannot close input file return case SRATE printf Incompatible sampling rates for files return 89 fee he eee ke ke ke kk bo ak aak akak eo SND NAME C skak ak akk ak e ak oc ac ak ale akk oie akk ak ok akk ak akk akk akk ak This program creates the first four digits of a file name for a recorded sound by using the standard time parameters of GMT MARKER EE ERA e ade ake HARARE ENEMA ae ake ak ake ake ake ale ak ake afe ake EAE ak ake HARE RA HAAN HA REA ak ake ak ak ake ak ak AACE char YR yr int yr switch yr case 90 return 0 break 1990 case 91 return 1 break case 92 return 2 break case 93 return 3 break case 94 return 4 break case 95 return 5 break case 96 return 6 break case 97 return 7 break case 98 return 8 break case 99 return 9 break case Ol return a break case 02 return b break case 03 return c break case 04 return d break case 05 return e break 2005 switch mn case 1 return l break January case 2 return 2 break case 3 return 3 break case 4 return 4 break case 5 return 5 break case 6 return 6 break case 7 return 7 break case 8 return 8 break case 9 r
69. p condition y n y cr Enter the table name officer cr Enter attribute name rank cr officer rank rank c20 Enter the condition Lt End group y n n cr Enter the table name officer cr Enter attribute name voice cr Enter the sound description soft voice cr End group y n y cr More condition n cr At this point the system converts the information given by the user into multiple SQL statements each of which is processed individually and the results coordinated and integrated to form the answer to the query Again the users are asked whether or not displaying the photos and playing the sound records is desired The detail information about this retrieval operation can be found Po90 26 B CATALOG DESIGN The design and management of the MDBMS catalog are different from the INGRES catalog management Since INGRES does not recognize the existence of media data we have to design and build the MDBMS catalog ourselves and manage the catalog as tables outside the INGRES The decision was made to create the catalog in the form of system tables in the internal memory throughout the operation of MDBMS When the user runs the MDBMS system the files which contain MDBMS catalog tables are read into memory before any user operation is performed and after the session the updated system tables are written out as files Pe90 Table Array Table List
70. r Wegener Dr C Thomas Wu Gregory R Sawyer and Cathy A Thomas who have made great contributions to the development of the MDBMS system and have thus provided me the opportunity to work on this important database project I would also like to thank Dr Kyung Chang Kim and my thesis companions Su Cheng Pei and Wuttipong Pongswuan with whom I worked together as a team to design the current MDBMS prototype I would like to express my sincere thanks to Dr Vincent Y Lum for all his support and encouragement in the conception and preparation of this thesis and his inexhaustible efforts in the development of the MDBMS project vii viii I INTRODUCTION A BACKGROUND Today in addition to standard data like numerics and alphanumerics it is possible to capture store manage retrieve and present different media information e g sound image graphics text and signals by using the current modern computer technology Multimedia Database Management System MDBMS beyond storing managing and retrieving different media information also manages the interrelationships among the data MDBMSs have found its way to many application areas like classroom teaching military command control and training libraries archives communication publishing advertising and computer integrated manufacturing Lo88 These application areas are nonstandard for the conventional DBMSs because DBMSs have been developed to manage only the standard
71. re environment to execute the play file commands in the remote PC but after a while we realized that Excelan network software package does not have RSH server capability when RSH command is invoked from the SUN environment through Ethernet Finally we realized that we can use the socket model which is also available in both environment 41 The socket interface model allows communication between an application program running on the system and the network Specifically this process is done in our PC between the running application program in the PC and the LAN Workplace TCP IP Transport System software running on the EXOS 205T Intelligent Ethernet Controller board which is already connected to the network In the SUN environment this process is automatically done by the 4 3 BSD UNIX operating system which is incorporated in SUN operating system by using available socket library routines as a feature of UNIX operating system In both environments the socket abstract model works in similar way Sockets are end points to which connections can be attached from the network and to which processes can be attached from the user in an application program The socket system works with the TCP IP and has nothing to do with low level structure of the network Ta88 The socket library provides for various functions such as reading writing accepting etc The socket functions are written in high level language C so it is easy to integrate the so
72. re insertion the insertion process is different than that for the formatted data The program needs the media data file name which includes the registration data and the raw data to fetch the registration and raw data and to put this information into the media data relation As explained in 90 the media relation is created by the system and is hidden from the user To complete the third part of the media object the description data the user has a chance to see the image or to listen the sound record before inserting the description in natural language form Again after the completion of all the attributes the system displays the inserted attributes values to get a confirmation from the user and gives the user a chance to change the values if so desired The retrieve operation is used to get the required attribute values of the tuples with respect to the user defined conditions which coincide with the WHERE clause in SQL statements First the user enters the table names which relate to his her query If the users has more than one relational tables then he she has to enter join conditions to connect the tables Next the user enters the attribute names for each relational table The system checks each attribute names entered for their existence in the given table and 16 displays a warning message in case of a non existing attribute The next step is to enter the conditions to retrieve the desired information from the related tabl
73. s_encoding print DURATION f n hdr gt s_duration printf NRESOLUTION d n hdr s resolution printf nPlease press ENTER to continue a getchar break case 0 printf NnNaNINNNCSSekkekeololeie ale ake ake ak ak ahe ahe ake ale ake ak ak ake ake ake ake ake ak ake ake ake ak akk ak ak A n printf NAN THANK YOU ie printf n i HAVE NICE DAY n printf break end switch end while ans not in 0 1 j end while ans not 0 end main AR ke ke ke ak ake ke ae ahe ak ke je ke ke ak ake ake ake e ke ke e TIT ate 3h TAR RIT RRA ake ahe ae ARTA ATOR AT AA ATC 101 S Soho ke obe obe obe ke ee ake abe SND STR QC 5k eek a ake ak ak ak e ake ake ak akk akk akk akk ake ak ake ak 2k This structure represents the sound object header information which is transferred from PC to SUN environment to store the sound object registration information to the MDBMS tables 2e ake ak ake ak ske ehe ok eie dee ake ae ake ake ak ake ake ak ahe ae ake ake ake ake ake oe ake ak ERE ake ake ale ak ake ak ahe ak ake ake kk struct SND_HDR char sfname 13 int s size int s samplrate int s encoding float s duration int s resolution hdr info unique sound filename number of bytes 8K or 16K per sec 0 none 1 ADPCM time in sec and hundredths bits per sampie
74. se press ENTER to continue Note that to play a sound file from the PC end a user only needs to give the text file name e g yavuz to the system This however cannot be done from the SUN workstations There the full sound file name e g 08dh5255 snd must be given Such is the case as shown in the example at the end of this chapter This has been done because oniy programs not users will invoke the command to play sound files from the SUN environment After recording the new sound files and listening them for verification purposes if desired the next steps are to exit the PC sound management system and to transfer each sound text file to the SUN workstation We use ftp function to transfer these files in the Ethernet network in the following order for the current system fip virgo lt cr gt Remote User Name mdbms lt cr gt Remote Password xxuuux cr ftp cd sid cr ftp put yavuz cr ftp quit cr When the user is done with the sound file transfer the next issue is to insert these sound data into the MDBMS databases We have already talked about this in the previous chapter and will not be repeated here Now we will discuss the sound management integration interface by using a program separate from the MDBMS to be used for the test purposes for the PC SUN linkage and the sound management integration The routines of this program have been included in the main program of the MDBMS prototype
75. se SUN 3 Workstation did not have the sound processing capability at that time The content search for the media data was first implemented by Meyer Wegener LM90 by using a parser implemented by another faculty in the department to parse the natural language descriptions entered by user and get the result in an acceptable form by Prolog The initial parser was relatively simple and could parse only phrases Subsequently the parser was greatly modified to have the capability to parse complex sentences and structures Du90 The first implemented MDBMS prototype constructed on top of the INGRES DBMS was designed for managing only the image media data types in the SUN 3 Workstation environment and did not have the capability to process both formatted data and multimedia data It could retrieve the image media by using the identifiers of the image data or their natural language descriptions The current MDBMS prototype is to broaden the database handling capability of the system by providing the integrated support for both formatted data and multimedia data Its design and implementation is based on the same architecture of the previous 10 work and it will provide the high level operations of table creation and data insertion retrieval deletion and update for both formatted and multimedia data Three thesis students worked on these high level operations Pei Pe90 worked on the table creation and data insertion Pongswuan Po90 worked
76. section is related to the sound file creation and remote sound record playing the part related to the MDBMS is already discussed in the previous chapter When a user wants to create a sound file the record batch file is to be run in the PC After running this batch file the user gets the main menu as shown in Figure 11 The first option is the record sound operation which is used to create a sound record with the registration data from a standard input connection This operation also 43 PC SOUND MANAGEMENT SYSTEM 1 Record Sound 2 Play Sound 0 Exit Enter your choice number Figure 11 The main menu of PC Sound Management creates a text file containing the sound file name and the registration data to be used in the MDBMS The system asks the user to enter a file name for the text file which will be transferred for setting up the registration data from PC to the SUN environment The user does not need to know the long and complicated unique sound file name created by the system this name is hidden from the user Now we can go through the user interface of the record sound operation in the following order the italics represents the user s responses 1 cr RECORD SOUND Please ENTER a file name to transfer the sound text file to SUN yavuz lt cr gt Please ENTER 0 for sampling rate 8 Khz 1 for sampling rate 16 Khz 0 lt cr gt Recording in progress Press lt lt SPACE BAR gt
77. ssing Society of Japan pp 123 134 1986 Bertino E Rabitti F and Gibbs S Query Processing in a Multimedia Document System ACM Trans on Office Information Systems v 6 no 1 pp 1 41 January 1988 Chrisodoulakis S Theodoridou M Ho F Papa M and Pathria A Multimedia Document Presentation Information Extraction and Document Formation in MINOS A model and a System ACA rans on Office Information Systems v 4 no 4 pp 345 383 October 1986 Dulle J The Scope of Descriptive Captions for Use in a Multimedia Database System Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California June 1990 Elmasri R and Navathe S B Fundamentals of Database Systems pp 504 The Benjamin Cummings Publishing Company Inc 1989 Kosaka K Kajitani K and Satoh M An Experimental Mixed Object Database System in Proc IEEE Cs Office Automation Symposium Gaithersburg MD April 1987 IEEE CS Press order no 770 pp 57 66 Washington 1987 Lum V Y and Meyer Wegener K A Conceptual Design for a Multimedia DBMS for Advanced Applications Report no NPS 52 88 025 Naval Postgraduate School Department of Computer Science Monterey California August 1988 Lum V Y and Meyer Wegener K A Multimedia Database Management System Supporting Content Search in Media Data Report no NPS 52 89 62 LM90 Lo88 020 Naval Postgraduate School Departmen
78. system asks to the user whether to display the media data or not in a sequential order The delete operation is used for deleting the specific tuples in a table by entering condition statements or deleting all tuples and afterwards dropping the relation without 17 any condition Constructing the condition statements is the sa Xx as explained above for the retrieve operation The last one the modify operation allows the user to perform various operations to update tables and attribute names like changing the table names or the attribute names in the table to insert new attributes to the table to update the formatted data inserted previously by entering the conditions and then the attribute names the values of which the user wants to change and to update the media data by entering the media file name after specifying the conditions and media attribute name After changing the media file name the user has an opportunity to update the description of the media data as well After the completion of each high level MDBMS operation the system displays the main menu on the screen The user can select either another operation or the quit option to exit the system 2 MDBMS Applica ions Now we will go through the user interface design of the MDBMS by using the create table insert tuple and retrieve operations which are already implemented For example we want to create a relational table SHIP with the attributes S NAME TYPE S NO YR
79. t of Computer Science Monterey California March 1989 Lum V Y and Meyer Wegener K An Architecture for a Multimedia Database Management System supporting Content Search in Proceedings of the international Conference on Computing and Information ICCI 90 Niagara Falls Canada May 23 26 1990 Lockemann P C Multimedia Databases A Paradigm and Architecture Report no Nps 52 88 047 Naval Postgraduate School Department of Computer Science Monterey California September 1988 MLWS88 Meyer Wegener K Lum V Y and Wu C T Managing Multimedia Data Pe90 Pe91 Po90 5 88 5 91 Ta88 An Exploration Report no NPS 52 88 010 Naval Postgraduate School Department of Computer Science Monterey California March 1988 Also published in Visual Database Systems Kunii T L pp 497 523 The North Holland Publishing Company Inc 1989 Pei S Design and Implementation of a Multimedia DBMS Catalog Management Table Creation and Data Insertion Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California December 1990 Peabody C Design and Implementation of a Multimedia DBMS Graphical User Interface Design and Implementation Master s Thesis Naval Postgraduate School Department of Computer Science Monterey California in preparation Pongsuwan W Design and Implementation of a Multimedia DBMS Retrieval Management Master s Thesis Naval Postgr
80. tegration to main MDBMS program are included in the Appendices A and B respectively SURVEY OF THE MDBMS A DEVELOPMENT OF MDBMS The work in the MDBMS Project at the Computer Science Department of the Naval Postgraduate School started with the design of the architecture of the MDBMS in 1988 LM88 LM89 The main goal was to have a database system that can process the multimedia data as conveniently as the processing of the standard structured data The architecture of the system is composed of the three main parts the MDBMS User Interface which is the interface between the MDBMS user and the formatted and media data managers the Standard DBMS which manages all the queries related to the formatted data and the Multimedia Manager which deals with the multimedia data One may consider that the Multimedia Manager to be composed of different subsystems related to the various media data types as image sound text graphics and signals The current MDBMS includes only the mage Manager and Sound Manager since only image and sound data are supported at this time The architecture of the system is shown in Figure 1 and the detail discussion about the MDBMS architecture is presented in LM88 Media data management requires the processing of both the structured and unstructured data together because it is not possible to handle media data by itself For multimedia data processing the abstract data type ADT concept has been determined
81. ter Science Software Technology P O Box 500 500 D 4600 Dortmund 50 GERMANY 104
82. the hierarchical tree structure to process a media As a result the users might easily get lost during a process Additionally hypertext requires an interpreter to process the user commands Furthermore the hypertext and hypermedia data cannot be accessed by the other users as in he database systems because they are designed to work only on personal computers in the single user environment MDBMS which is a DBMS introduced in LM89 with the extended capability to process the multimedia data was designed to overcome the restrictions and disadvantages of hypertext and hypermedia systems B AN OVERVIEW MDBMS mainly consists of two subsystems one to manage the structured data in the conventional database environment NGRES and the second one to process the multimedia data in an extended DBMS environment These two subsystems are integrated to provide a uniform interface to process structured or multimedia data or both together The general design of the overall application for MDBMS includes the main high level database operations like table creation data insertion retrieval deletion and modification Three students are doing related work on their M S theses in the Computer Science Department of the Naval Postgraduate School The detail design and implementation for table creation and tuple insertion along with catalog management are presented in the thesis by Pei Pe90 The design and implementation for retrieval operation can be
83. the words or phrases as defined in this dictionary to write the description data The parser transforms the description data entered by the user into a set of predicates and literals suitable for Prolog processing These predicates state a fact about the content in the media object When the user enters a query by using a media content description like officer with glasses the system calls the parser to perform a content search throughout the media data descriptions Three M S thesis students worked on the design and implementation of the current MDBMS prototype After the collective work on the design part the detail design and implementation for table creation and tuple insertion with catalog management were done by Pei Pe90 and the design and implementation for the retrieval operation were done by Pongswuan Po90 In this thesis we accomplished the sound management integration into the system which includes the storage and management of the sound media data using an IBM compatible personal computer the connection between the MDBMS and the PC and the integration into the MDBMS 59 The user can record and play sound data in the PC During the sound recording process two different files are created one with the registration data and the bit string of sound data with a system created unique file name and the other one a text file containing the unique sound file name and the registration data belongs to the sound record with a user
84. the design and implementation of MDBMS Po90 Figure 3 The MDBMS System In the meantime the capabilities of INGRES and SUN workstations have changed Now some of the INGRES restrictions mentioned above have been removed and SUN can support sound As the MDBMS prototype is not intended to be a production system at this time a decision was made not to change the structure of the system because the current system is enough for the demonstration of the various concepts Otherwise it would require substantial investment to purchase new hardware and recode some written software It was decided that instead of these investments which would provide little gain the IBM compatible PC system would be retained to manage sound data and would 12 be integrated into the MDBMS prototype in SUN environment as a backend server for sound management via a local area network connection which in this case is the Ethernet For image capturing process a video card which works with a camera recorder was installed in a PC After capturing the images the image files are transferred into a SUN workstation and processed The system environment just discussed influences the design and implementation of the system and will be reflected in the various parts in this thesis as well as the theses by Pei Pe90 and Pongsuwan Po90 However the complexities existing in the system s configuration are transparent to the users They will find all th
85. three arrays for character integer and float data types and two record arrays for media data image and sound representing the data types that the MDBMS supports The value_Array tables are used to store the data before it is inserted to the database and also before it is displayed to the user C_Value I Value F Value char int float Figure 8 MDBMS Value_Array tables Now for illustrating the use of the catalog tables let us use the two relations named SHIP and OFFICER which we have used for the user interface application in the previous section First when a new relation is to be entered the catalog management part of the MDBMS will search through the relation names in Table_Array table checking if a duplication exists for the new relation name with the previous relations In the case of any iable name duplication the user is given an opportunity to insert a new relation name If no duplication exists the catalog manager will put the relation into the next available row in the Table Array table and transfer the row number of the new relation to the Table List array For instance when we create the new relation OFFICER after the relation named SHIP the system assigns the second row for our new relation after checking for table name duplication After that the row index number in this case 2 is transferred to the next available slot in the Table List array as shown in Figure 6 The following step is to enter
86. tially consists of three parts registration data raw data and description data as shown in Figure 2 The registration data is necessary to interpret or display identify or distinguish a definite raw data from others The registration data may have a fixed format because the formats or field lengths of information required to access the different media data are known The description data that describes the object representation of the raw data and is used for content search purposes generally can not be derived by computers and must to be entered by the user in natural language form understand these descriptions the system requires a fairly rich and sophisticated dictionary depending on the application An application naturally limits the scope of this dictionary size since vocabulary not used in the application should not be entered The image and sound media data types are already defined Th88 Sa88 Both have their own sets of operators or functions for processing the registration data the raw data and the description data The registration data in the image ADT include file size resolution encoding and colormap and the registration data in the sound ADT include file size sample rate encoding duration and resolution Processing description data requires sophisticated techniques of natural language processing A parser was thus constructed for this purpose Using the dictionary or lexicon prepared for the application transfor
87. tions has the attributes image id file id description height width and depth and the sound media relation has sound id file id description file size sampling rate encoding technique duration and resolution In both media relations the image id and sound id attributes are defined by the system internally and used by the system as a link between the main relation and these media relations The second attributes file id contain the exact path where the files which include the registration data and raw data belong to each media exist in the MOBMS system environment like n virgo mdbms snd The description part is entered by the user in the natural language form as explained in the previous section The rest of the attributes are extracted from the media file and are used to display the image or to play the sound record The media relation tables created by INGRES are shown in Figure 9 The detail information about the image media can be found in Po90 Th88 and we will present the detail information about the sound media and its interface with the MDBMS system in the rest of the chapters as the emphasis area of this thesis Figure 9 The Image and Sound Relational Tables 32 For the case of inserung tuples to the created relational tables the insertion of the formatted data into the database can be done easily by using INGRES but for the media data case the procedure is different The media files co
88. truct SND HDR hdr hdr struct SND HDR malloc sizeof struct SND HDR s gets the remote name like pclum1 or pclum2 to play sound printf nNPlease ENTER lt lt the remote PC name gt gt to play sound file gets pcname while answer 0 answer 0 while answer lt 0 Il answer gt 2 print hnhWwINNNTHE SUN REMOTE SOUND PLAY MANAGEMENT n printf Ng 3 a ate ak ahe atea sde ak ate abe abe abe ahe abe hee de te ede pee kk print f oNX1 Play Sound File PC print nNN2 Display Sound File Header Information printf hnNNO Exit print printf n tEnter your choice number answer getchar while getchar printf nYour answer is c n answer Switch answer case 1 printf Nn t PLAY SOUND IN PC n puts Enter the sound filename xxxxxxxx snd to play in PC n gets filename SendSocket filename pcname play sound file in remote PC a getchar break case 2 printf n Wt DISPLAY SOUND FILE HEADER INFO n n puts Enter the USER DEFINED sound text file name n gets filename snd inforead filename hdr read the header info returned header info from snd info read function printf hN FILENAME s n hdr gt sfname printf A SIZE d n hdr gt s_size printf NSSAMPLERATE d n hdr gt s_samplrate printf NENCODING d n hdr gt
89. ts if read snd hdr filename amp hdr 0 vpfunction BEGIN alert to the driver port 0x280 useint 2 VP620 amp vpfunction amp useint amp port if hdr s_samplrate 8000 srate 0 else srate 1 vpfunction PLAY VP620 amp vpfunction amp srate filename open file vpfunction STATUS do VP620 amp vpfunction amp overload amp hundsec amp sec amp error amp state while kbhit amp state 3 these statements always required to close the file vpfunction STOP VP620 amp vpfunction vpfunction END VP620 amp vpfunction return ERROR FREE else displayerr READ return SOUND ERROR Se EERE ER REE REE READ FILENAME 3 kak xk ak ak ak ak kk ak ak ak ok 2 ake ok ke ake ake ake ale ake loko This function reads a filename from the playfile file which is created by the receiver socket xke ake ae ake ake ok ahe afe ake ake A 3k ake ke ahe de ke ake ahe ke joke ake ake e a afe ake ake ake ook 34e ahe afe ale ake ake ake ake ahe ae IOC ake ale IO III OR ake 3k ake Read_filename fi ename rpfname char filename output char rpfname remote playfile name FILE f if f fopen rpfname rb NULL open for reading displayerr ROPEN return SOUND_ERROR read the filename from the predesignated input file if fread filename 12 1 f 1
90. ue supanoup Multimedia Database Management System Multimedia DBMS MDBMS Sound Media Management Sound Database 19 ABSTRACT Continue on reverse if necessary and identify by block number me Today in addition to standard data like numerics and alphanumerics it is possible to capture store manage retrieve and present different media information e g sound image graphics text and signals by using the cur rent modern computer technology The Multimedia Database Management System MDBMS project which started at the Computer Science Department of the Naval Postgraduate School in 1988 studies not only the storing managing and retrieving different media information but also the management of the interrelationships among the data This thesis concentrates on the Sound Management Integration of the MDBMS prototype which includes the storage and management of the sound records in an IBM compatible personal computer and the connection and integration of it to the database system in the SUN environment through a local area net work after presenting the general overview of the MDBMS the system environment the user interface and the catalog designs 22b TELEPHONE include Area Code 22c_OFFl YMBOL 408 646 309 OD FORM 1473 84 MAR 83 APR edition may be used until exhausted SECURITY CLASSIFICATION OF THIS PAGE Ail other editions are obsolete i UNCLASSIFIED Approved for public release distribution is unl
91. und management and the program code is included in Appendix A B SOUND MANAGEMENT INTEGRATION INTO MDBMS In the MDBMS environment we need to read the sound text file which is sent from the PC end of the system write these information into the sound relational tables Figure 9 add description data as a third part of the sound media object during the tuple 55 insertion operation as stated earlier and finally send the unique sound file name to the PC by using a sending socket to play a sc ad record We use the following programs to illustrate how the sound management is integrated into the MDBMS SDEMO C SUN sound demo SND STR C sound structure SDEMO C is the main program and uses the module SND STR C The module SND STR C is different from the SND_STRU C because we added the unique sound file name attribute in addition to the header information in this structure SDEMO C sends a sound file name to the PC to play sound record by using a sending socket and displays the information in the sound text file This program first asks for the remote PC name which has sound processing capability to play sound records before displaying the main menu There are two main functions in this program Sendsocket and Snd inforead SendSocket is a sending socket function and is a modification of the Internet domain stream connection program in the SUN Microsystem Network Programming manual It sends the remote sound
92. version of sampled energy levels into numerical quantities is known as digitizing S288 If the samples are sufficiently close together in other words if we increase the rate of sampling the digitized sound record is heard to be spatially continuous like the original sound We define the sampling rate in Hertz Hz for example 8000 samples per second corresponds to 8000 Hz or 8 Khz On the other hand when we increase the sampling rate the size of the digitized sound 35 data that we store in the memory will also increase So there is a trade off between the quality of the digital sound record and the file size required to store the sound The encoding techniques are used to reduce the number of bits required to store the digitized sound We will discuss the encoding techniques related to our system During the encoding process the systems filter and digitize the analog signal analyze short segments of it then encode prior to transmission or storage The encoding technique that we use in our system is adaptive differential pulse code modulation ADPCM This technique is included in the waveform encoding category and is an extension of pulse code modulation PCM In PCM the amplitude of the analog sound signal is sampled at a fixed rate and converted into digital information using an ADC On the other hand the differential pulse code modulation DPCM uses a different technique and stores the difference between the current value

sound management integration

Contents

Download Pdf Manuals

Related Search

Related Contents