Home

Music training apparatus

Contents

1. AC COUPLED HALF CROSS WAVE DETECTOR RECTIFTER GATE DETECTOR RESET ENVELOPE FOLLOWER TO 56 OTHER Fig 3 SPECTRAL 11 ESTIMATOR U S Patent doni T 5 287 789 FILTER RECTIFIER N 105 SPECTRAL 1202 ESTIMATOR an TO OTHER ENVELOPE SPECTRAL FOLLOWER p ESTIMATOR U S Patent Feb 22 1994 Sheet 5 of 7 5 287 789 107 309 310 213 VHIGH d 74 N f N J M 4 7 M 212 VLOW f 308 GROUND U S Patent Feb 22 1994 Sheet 6 of 7 5 287 789 US Patent Feb 22 1994 Sheet 7 of 7 5 287 789 502 VIDEO KARAOKE PLAYER aubo 5 287 789 1 MUSIC TRAINING APPARATUS BACKGROUND OF THE INVENTION This invention generally relates to a teaching device to assist music students in recognizing and producing accurate pitch timbre tonal quality and timing me ter on their musical instrument and more particularly to an electronic apparatus to quantify and provide vi sual feedback of the musical performance of a student to that of a musical reference A student of music for purposes of description is anyone who is trying to play a musical instrument The invention disclosed herewith discusses musical instru ments that produce a tone of detectable pitch This includes
2. tive voltage of the resulting signal 107 to the rating of the comparators 204 210 and 211 e g 0 3 volts for the National Semiconductor LM324 Zero Cross Detector 102 The zero cross detector 102 is a comparator 204 whose output 205 is high when the post filtered audio signal 107 goes above ground Refer ring to FIG 6C waveform 107 represents the audio signal applied to the zero cross detector 102 Waveform 205 represents the output of the zero cross comparator 204 When the input waveform 107 is above the ground reference 308 the comparator 204 output 205 is low The zero cross detector s output 205 is applied to the capture input 207 of the MCU 50 A rising edge on the capture input 207 causes the value an internal free run ning high speed counter e g 1 MhZ to be latched into an internal register and a software interrupt to be gener ated inside the MCU 50 An interrupt routing in the MCU 50 reads the internal register and calculates the period of the applied signal 107 by subtracting the latched value from the previous latched value Fre quency is the reciprocal of period Since our ears per ceive pitch logarithmically a preferred embodiment of the invention converts period into the logarithmic fre quency units of semitones and cents hundredths of a semitone where a semitone is a frequency increment of the factor the twelfth root of two This conversion may be approximated by finding the nearest semitone by searc
3. 1 wherein the means for providing an audible tone reference comprises a stereo recording having one channel carrying audible tones as the monophonic reference source and another channel carrying the audible tone reference 23 The apparatus of claim 22 wherein the audible tones of the monophonic reference source comprise solo singing 24 The apparatus of claim 1 wherein the computer means further includes means for representing as two different lines of different brightness on the display means the sequence of pitch values from the monopho nic reference source and the series of intonations made by the student 25 The apparatus of claim 1 wherein the monopho nic reference source comprises a pre recorded se quence 26 The apparatus of claim 1 wherein the computer means and display means include means for visually displaying musically related pitches to the monophonic reference source 27 The apparatus of claim 1 wherein the monopho nic reference is stored on audio tape 28 The apparatus of claim 1 wherein the monopho nic reference is stored on compact disk 29 Apparatus for determining the pitch of an audio input signal comprising filter means for attenuating the harmonics of the audio input signal means for converting the output of the filter means into a pulse train of frequency substantially equal to the output of the filter means a control means responsive to the droop of the out put of the filter means f
4. 45 50 55 60 65 18 control routine examines state of DROOP 237 LOW 238 and HIGH 239 and takes the action set forth in TABLE 3 below TABLE 3 DROOP 237 LOW 238 HIGH 239 ACTION LOW LOW LOW increase cutoff frequency HIGH X X decrease cutoff frequency LOW HIGH LOW remain the same do nothing LOW HIGH HIGH decrease the cutoff frequency X Don t care An action of increase means the cutoff frequency is increased by an increment decrease means the cutoff frequency is decreased by an increment and remain the same means the cutoff frequency is not changed The cutoff frequency increment used is dependent on the characteristics of the filter 103 and DAC 104 For exam ple large increments would be used with a filter 103 having a long settling time Steps on the order of 1 to 6 semitones are typical If the advanced pitches of the monophonic reference are known as in the case of a note event list the cutoff frequency of the filter for the student may be guessed in advanced based on upcoming pitches Low Pass Filter 103 There are many ways to imple ment a tunable low pass filter For example a bank of fixed cutoff filters can be selected with an analog multi plexer e g National Semiconductor CD4051 or the active components of the filter e g capacitors and resistors that determine the cutoff frequency may be electronically changed with a multiplexer e g Na tional Semiconductor CD4066 Other examples may
5. Audio lines 47 and 48 from the output of the audio tape player 17 are routed through a channel switcher 51 a double pole single throw switch set by the control SWITCH CHANNEL 31 This allows either output line 47 or 48 of the audio tape player 17 to be treated as monophonic reference signal 52 and the alternate as an accompaniment signal 63 The monophonic reference signal 52 is applied to a command tone decoder 53 which detects the presence of a high frequency marker tone e g 10 kHz used to encode commands in the monophonic reference signal 52 A low pass filter 54 removes the marker tone from the rest of an audio path 55 The filtered monophonic reference signal 55 is applied to a waveform analyzer 56 a multiplexer 57 and an audio mixer 59 The wave form analyzer 56 extracts the aural parameters pitch amplitude duration and timbre An audio digitizer 58 converts a continuous audio signal 99 into digital sam ples 60 The multiplexer 57 selects an audio signal 55 or 63 to be digitized typically the monophonic reference signal 55 The audio mixer 59 combines the monophonic reference signal 55 with other audio signals the sum of which are amplified by an audio amplifier 61 and heard on the headphones 15 The multiplexer 57 is normally set by a control line 62 of the MCU 50 to select the monophonic reference signal 55 as the source to be digitized The audio tape 16 can instruct the MCU 50 through a tone encoded com mand detect
6. It is helpful for a student of music to see several notes in advance in order to plan playing technique necessary to shape musical phrases None of the musical system mentioned display tones in advance of being heard Of the numerous musical instruments a student might want to learn singing is often the most psychologically difficult for those adults who were told as children that they could not sing These adults are often reluctant to attempt singing in front of others for fear of judgement Singing is a skill like reading that needs to be devel oped by instruction and practice Individual instruction is often necessary for each student s errors and progress are unique Typically vocal instruction requires finding a music teacher arranging a visitation schedule paying for the classes and maintaining regular attendance These factors can discourage potential music students from pursuing instruction An ideal music instructor would be available anytime anywhere would have infinite patience be consistently accurate non judg mental could be shared among several people for no additional cost provide instruction on any of a thou sand popular songs show exactly where a student s 15 25 30 35 40 45 55 60 65 4 errors are and comply with the interests and pace of each individual student It can be seen therefore that a need exists for a music training apparatus that can provide a student with an accurate te
7. The system of Nickerson et al displays a time history of pitch duration and timbre but is not well suited for musical instruction The system uses a miniature accel erometer applied to the throat with adhesive tape to measure the pitch of the student s voice Since the stu dents are deaf no consideration for aural reference is made Data collected is presented in the context of speech not music and no provisions are made for pitch tracking musical instruments Producing an accurate static tone is a good start for a music student however music is the dynamic organiza tion of sound over time An accomplished musician needs additional skills to produce a sequence of tones playing a melody match a sequence of tones playing a melody in key produce a tone relative to a reference tone playing an interval produce a sequence of tones relative to a sequence of reference tones playing in harmony produce tones in a broad range of pitches range quickly varying the pitch and amplitude vi brato and tremolo produce tones at specific times and durations playing in meter and produce tones of good timbre tone quality Neither Mercer or Tumblin have the display neces sary to show a time history Mercer has two pitch track ers but lacks any memory means to store the pitch data Tumblin has music exercise data stored but only has one pitch tracker and does not store the pitch data Tumblin uses music exercise data that must be sp
8. and strength of the student s intonations An indicator lamp not shown may be included to indicate when gain of the audio pre amplifier 64 was too high i e the amplifier has reached its maximum output and is distorting The MCU 50 decodes commands from the tape player 17 by monitoring the output of the command tone decoder 53 The MCU 50 also receives commands from the front panel control switches 18 5 287 789 11 The MCU 50 saves the most recent samples of digi tized audio e g the last 8 seconds worth from the audio digitizer 58 in RAM 67 The most recent samples of aural parameters from the waveform analyzers 56 and 65 are saved in the host computer 13 When the LOOP switch 27 is pressed by the student the MCU 50 stops the audio tape player 17 and sends the last several seconds of digitized audio samples in a continuous endless loop to the DAC 68 The DAC 68 converts digital signals 73 into an audio signal 69 that is mixed with the voice of a student 72 by the audio mixer 59 amplified by the audio amplifier 61 and heard on the student s headphones 15 For simplicity the combined audio signal 46 is considered monophonic sum of all the audio signals applied to the audio mixer 59 and heard equally on each headphone 15 of the headset 11 It is understood that additional controls commonly occurring on audio mixers would provide a stereo out put with individual spacial placement pan or balance control for each of the a
9. and system ground a resistor connected between comparator out put and second input and means for applying said waveform to first input the output of the comparator responsive to the change in slope of said waveform 15 The apparatus of claim 1 further including means for digitally storing in the computer means a brief his tory of the audible tone reference and of the visual display relating to the monophonic reference source for a selected interval for replay audibly and on the visual display means as desired 16 The application of claim 1 wherein the audible tone references comprises a polyphonic music source 17 The application of claim 1 wherein the monopho nic reference source comprises a note event list 18 The application of claim 17 wherein the audible tone reference comprises a series of tones produced from the note event list 23 19 The application of claim 17 wherein means for providing an audible tone reference comprises a stereo audio tape recording having one channel with the note event list as the monophonic reference source and another channel carrying the audible tone refer ence 20 The apparatus of claim 17 wherein the monopho nic reference source further comprises encoded lyrics 21 The apparatus of claim 17 wherein the monopho nic reference further comprises encoded computer commands with means for causing further features to be displayed on the display means 22 The apparatus of claim
10. attenuated by a tunable low pass filter and applied to a zero cross detector The cutoff frequency of the low pass filter is dynamically determined by analyzing fea tures of the output waveform of the low pass filter The pitch durations of the tone source may be determined by detecting abrupt changes in the frequency ampli tude and or spectrum of the tone source In a preferred embodiment a fixed amplitude threshold is used to detect duration an envelope follower is used to detect amplitude and counting the number of times the slope of the tone s waveform changes sign is used to deter mine timbre Said means to detect pitch amplitude duration and timbre are used in a preferred embodi ment to determine the aural parameters of student into nations and that of audible monophonic reference sources The absolute pitch of a musical interval is based on a musical scale built on a reference pitch tonic In a preferred embodiment the tonic is provided by the monophonic reference source Intervals may be visually represented in many forms In one embodiment inter vals appear as horizontal lines vertically positioned by pitch In one embodiment aural parameters are digitally stored in random access memory RAM by the opera tion of a microcontroller unit Aural parameters from one or more sources may be visually represented in a variety of ways One embodiment visually repre sents aural parameters on a graphic display
11. compact disk and producing an image visually representing a sequence of pitch values the monophonic reference source comprising de sired pitches to be achieved by the student means for providing an audible tone reference to the student in unison with the monophonic reference source and sharing the same fundamental sequence of tones as the monophonic reference source pickup means for picking up intonations made by the student in attempting to follow the audible tone reference pitch tracker means receiving a signal from pickup means for analyzing and tracking the pitch of the student s intonations and for generating a pitch track signal and computer means for receiving pitch track signals from the pitch tracker means and for producing an image visually representing a temporal history of the series of intonations made by the student image mixing means for combining a plurality of images to be viewed simultaneously by the student on the visual display means where the plurality of images include the visual image of the sequence of desired pitches the temporal history of student intonations and the compact disk visual images 36 The apparatus of claim 35 wherein the monopho nic reference source comprises an audible tone and including a second pitch tracker means receiving the monophonic reference source the second pitch tracker means generating a signal to the computer means which in turn is used to generate the image visually r
12. generating a signal to the computer means which in turn is used to generate the display of the monophonic reference source pitches on the visual dis play means 3 The apparatus of claim 2 wherein the audible tone reference includes the monophonic reference source 4 The apparatus of claim 2 wherein the monophonic reference source comprises a live unrecorded intona tion 10 20 25 30 40 45 50 55 65 22 5 The apparatus of claim 1 wherein the computer means includes means for presenting the student s into nations and the desired sequence of pitch values in a scrolling format moving across the display means 6 The apparatus of claim 5 wherein the display means includes monophonic display means for display ing the pitch of the monophonic source as a line whose vertical position increases with the pitch of the mono phonic source 7 The apparatus of claim 6 wherein the computer means and the display means include pitch display means for displaying the pitch of the student s intona tion as a line whose vertical position increases with the pitch of the student s intonation in a format visually similar to the line displayed for the monophonic refer ence source 8 The apparatus of claim 1 further including means associated with the computer means and the visual dis play means for displaying in advance a section of the desired sequence of pitch values in a display format visually similar to dis
13. sharp could be viewed on the display screen 22 Further controls may be provided to tune and trans pose the audible tone reference when provided by a tone generator under control of the invention A simplified block diagram of the control unit 12 is set forth in FIG 2 A microcontroller unit 50 MCU 68HC11 Family Microcontroller Motorola Inc Mi crocontroller Division Austin Tex 78735 is used to perform much of the control functions of the invention 10 Although this microcontroller with its built in ana log to digital converter ADC electrically erasable read only memory EEPROM random access memory RAM read only memory ROM serial communica tion interface SCI 16 bit timer and pulse accumula tion circuit PAC provide for a highly integrated 20 25 30 35 45 55 small inexpensive and low power implementation of 60 the invention 10 it is not required and may be replaced with other electronic circuitry or software For simplic ity however the invention will be descried using the microcontrol unit MCU 50 as the controlling device The function of many of the pins on the 68HC11 are software configurable so their designated names do not always disclose their function For clarity pins on the MCU 50 have been renamed according to their function 65 10 as follows DIN data in DOUT data out ADC analog to digital converter RCV serial in TX serial out PAI pulse accumulator input
14. storing said parameters a means for visually representing a tem poral history of said parameters a means for providing reference aural parameters a means for determining and visually representing musical intervals of the monopho nic reference source a means for digitally recording and playing back an audio signal a means for providing audible tone reference and a means for providing musi cal instruction A monophonic reference source is the musical perfor mance the student is striving to emulate The perfor mance may be as simple as matching a single pitch singing intervals to a pitch or as complex as singing an entire opera The monophonic reference source con tains a sequence of one or more aural parameter which include pitch timbre amplitude and duration Mono phonic reference sources include a live or recorded series of tones produced by such sources as a human voice musical instrument or electronic device The monophonic reference source need not be audible The sequence of aural parameters a note event list may be provided as data organized on media such as mag netic disk tape or computer memory note event list is a chronological record of pitch and duration of a musical performance A piano roll is an example of a note list recorded on paper Note lists can also be cre ated by playing on a synthesizer and having a computer record the key strikes note events Libraries of pop ular and classical songs
15. the headphones 15 The student sees a visual representation of aural parameters of his voice and the monophonic reference channel on the display screen 22 REWIND 24 The audio tape 16 rewinds to review past material FAST FORWARD 25 The audio tape 16 advances to new material LOOP 27 The audio tape 16 stops The student hears the most recent history of the instructor s intonation digitally replayed in an endless loop and sees aural parameters of student s and instructor s intonations on the display screen 22 Alternate presses terminate the LOOP function STOP 26 The audio tape 16 stops The LOOP func tion if on stops SHOW INTERVALS 28 Musical intervals e g third fifth octave related to a tonic are visually repre sented on display screen 22 When the SHOW INTER 5 287 789 9 VALS switch 28 is pressed the current pitch of monophonic reference source is saved and used as the tonic If no pitch is present on the monophonic refer ence channel when the SHOW INTERVALS switch 28 is pressed the previously stored tonic is used Suc cessive presses of the SHOW INTERVALS switch 28 toggle the function on and off SHOW HARMONY 29 Harmony pitches are dis played e g third fifth and octave for each monopho nic reference pitch The tonic is determined by the same technique used in SHOW INTERVALS Successive presses of the SHOW HARMONY switch 29 toggle this function on and off DISPLAY SPEED 30 changes
16. to a crescendo identify rhythmic patterns view lyrics as can be coded in a note event list and otherwise prepare his instrument e g breath ing for a vocalist placing a bow for a violinist The block diagram of electronic circuitry as set forth in FIG 3 details the waveform analyzer 56 which quantifies the aural parameters of pitch amplitude du tation and timbre of the applied audio signal 55 The waveform analyzers 56 and 65 are identical and differ only in the signals that enter and leave them Therefore only waveform analyzer 56 will be examined 5 287 789 13 The pitch of the applied audio signal 55 is determined by measuring the period of its fundamental frequency The period is detected by a zero cross detector 102 The fundamental frequency is extracted from the audio source 55 by a tunable low pass filter 103 The cutoff frequency of the tunable low pass filter 103 is deter mined by a control input 130 voltage set by a digital to analog converter 104 DAC The MCU 50 adjusts voltage output 131 of the DAC 104 based on the output of a droop detector 105 and a gate detector 106 The droop detector 105 indicates the amplitude and ten dency of an applied waveform 107 to make more than two zero crossings during a pitch cycle which would result in a pitch tracking error The gate detector 106 indicates when the amplitude of the applied audio signal 55 exceeds a fixed threshold A latch 108 stores the momentary output o
17. tone generator may be used to express the tone of the aural parameters Several firms e g Pocket Songs 50 S Buckhout Street Irvington N Y 10533 SongTrax TM by Song tech 3900 E Whiteside Street Los Angeles Calif 90063 provide pre recorded popular music in a format multiplexed format that separates a solo instrumen tal performance from an accompaniment as in the ka raoke system Typically a stereo tape recording is pro vided with a solo vocal performance on the right chan nel and a band playing accompaniment on the left chan nel This multiplexed format can provide both the audi ble tone reference and monophonic reference source In one embodiment of the invention the solo vocal channel is used as the monophonic reference source and the combination audio mix of both channels is used as the audible tone reference The means for providing musical instruction may be any recorded material that guides the student This may include written material e g a text or workbook text and or graphics on a visual display audible material on a compact disk vinyl record or audio tape and mate rial digitally stored in RAM ROM optical disk or magnetic media For example the recorded material may guide the student by providing an audio reference of a properly played instrument e g a karaoke record ing or by providing a syllabus of instruction on applied music theory e g a lecture on scales and intervals with musi
18. 70 set by the voltage divider R13 and R14 The output 209 of the comparator 208 is stored in the latch 27 e g National Semiconductor 74LS279 Droop Detector 105 The droop detector indicates the magnitude and tendency of the applied audio signal 107 to make more than two zero crossings per pitch cycle The magnitude of the applied audio signal 107 is reported by the comparators 210 and 211 respectively The output 214 and 215 of comparator 210 and 211 goes low when the amplitude of the applied audio signal 107 exceeds their respective threshold VLOW 212 and VHIGH 213 The threshold VLOW 212 is set by the voltage divider composed of R15 and R16 The thresh old VHIGH 213 is set by resistors R17 and R18 The outputs 214 and 215 are stored in the latch 271 Referring to the waveforms of FIG 6C droop is the portion or portions of the waveform 107 above ground 308 where the slope is changing from negative to positive occurring at point 309 and 310 Droop is caused by excessive energy in upper harmonics which can result in more than two zero crossing per pitch cycle The droop detector s output 232 goes low after a droop 310 crosses the low threshold VLOW 212 set by the voltage divider R15 and R16 The droop detector 105 operates by counting the number of times the ap plied waveform 107 crosses the low threshold VLOW 212 during each pitch cycle The pitch cycle is deter mined by the zero cross detector 102 Referring to the electronic sch
19. GS FIG 1 shows a perspective view of the invention illustrating the basic system components FIG 2 illustrates schematically by means of a simpli fied block diagram the electronic components of the present invention FIG 3 illustrates schematically by means of a simpli fied block diagram the waveform analyzer FIG 4 illustrates schematically by means of a simpli fied block diagram the command tone decoder FIG 5 illustrates a circuitry detail of the waveform analyzer FIG 6 illustrates by means of waveforms the opera tion of various circuits in the waveform analyzer FIGS 7 to 12 illustrate visual representations of aural parameters used in the preferred embodiment of the invention FIG 13 illustrates schematically by means of a sim plified block diagram an application cf one embodi ment of the invention that records on video tape the performance of a student FIG 14 illustrates schematically by means of a sim plified block diagram an application of one embodi ment of the invention that produces a superimposed image of a karaoke player s video output with a tem poral visual representation of the singer s aural parame ters DESCRIPTION OF PREFERRED EMBODIMENTS For purposes of explanation and simplicity the inven tion will be described in the context of voice instruction using the intonations and speaking voice of a pre recorded vocal instructor as the monophonic reference source audible referenc
20. United States Patent Zimmerman US005287789A Patent Number 4 Date of Patent 5 287 789 Feb 22 1994 54 MUSIC TRAINING APPARATUS 76 Inventor Thomas G Zimmerman 218 15 Hartland Ave Flushing N Y 11364 21 Appl No 803 035 22 Filed Dec 6 1991 FSET Tint CU oue e ctione G09B 15 02 1521 WSs CL 84 477 R 84 454 58 Field of Search 84 470 R 477 R 454 56 References Cited U S PATENT DOCUMENTS 3 876 936 4 1975 Lester et al 324 79 R 3 919 913 11 1975 Shrader 84 454 4 063 035 12 1977 Appleman et al 179 1 SP 4 273 023 6 1981 Mercer 84 454 4 321 853 3 1982 Tumblin 84 454 4 378 466 3 1983 Esser 4 416 182 11 1983 Wise et al 4 429 609 2 1984 Warrender 84 454 4 580 133 4 1986 Matsuoka et al 4 969 194 11 1990 Ezawa et al 5 070 754 12 1991 Adamson 84 454 OTHER PUBLICATIONS Computer Aided Speech Training for the Deaf Journal of Speech and Hearing Disorders Feb 1976 vol 41 No 1 R S Nickerson D N Kalikow Niedzwiecki and Mikiel 1976 Hess Wolfgang Pitch Determination of Speech Signals vol 3 of Springer Series in Information Sciences Springer Verlag New York p 175 Users Manual Voicetracker by Fairlight In
21. be found in Chapter 9 of Don Lancaster s book Active Fil ter Cookbook For simplicity a 4 pole tunable low pass filter 103 is shown e g CM3320 Curtis Electromusic Specialties 110 Highland Avenue Los Gatos Calif 95030 The cutoff frequency of the tunable low pass filter 103 is determined by the voltage on the control input 130 set by the DAC 104 under the control of the MCU 50 The performance of the pitch tracker may be im proved by maximizing the amplitude without clipping applied to the low pass filter 103 This may be done by adding an automatic gain control AGC circuit at the input of the filter 103 An inexpensive DAC 104 is implemented with diodes 011 and 012 resistor R19 capacitor C12 and opera tional amplifier 219 e g National Semiconductor M324 When the DECREASE line 220 is high and the INCREASE line 221 is low the voltage on capaci tor C12 is maintained When the DECREASE line 220 goes low the voltage on capacitor C12 decreases as it discharges through resistor R19 and diode D11 When the INCREASE line 221 goes high the voltage on capacitor C12 increases as it charges through resistor R19 and diode D12 Operational amplifier 219 is config ured as a voltage follower to prevent the control volt age input 130 from loading the capacitor C12 The voltage output of the DAC 13 is monitor by the MCU 50 by the analog to digital channel 250 Envelope Follower 109 The envelope follower 109 determines the overa
22. cal examples f In a preferred embodiment of the music training sys tem an audio tape recording of a vocal instructor pro vides both the source of musical reference and musical instruction The audio tape contains vocal exercises that are used with the invention to practice such exercises as holding a steady tone singing in meter proper rhythm singing a melody and singing harmonies The audio tape contains verbal descriptions of each exercise into nations by the instructor complete songs with vocal and musical accompaniment and encoded commands to automatically control functions of the invention The display screen simultaneously shows a temporal visual representation of aural parameters of the student and the instructor A preferred visual representation of tone sources are graphic plots where the vertical axis is pitch in logarithmic units horizontal axis is time line thick ness is amplitude line color is timbre and line bright ness is used to distinguish multiple tone sources In a preferred embodiment instruction in singing intervals and harmonies is provided by displaying musi cally related pitches e g third fifth and octave to a reference pitch from the monophonic reference source The display speed can be increased to disclose precise pitch contour revealing such features as vibrato pitch sustaining accuracy and note approach e g overshoot or undershoot 5 287 789 7 DESCRIPTION OF THE DRAWIN
23. ctly equal to any note of the chromatic scale it is considered out of tune and is described as being sharp or flat higher or lower in frequency relative to the nearest note of the chromatic scale Three common intervals are the third fifth and eighth These refer to the third fifth and eighth note of the scale The eighth note is an octave interval twice the frequency of the tonic The pitch of the tonic must be known in order to determine an interval Since intervals are ratios musical calculations are best done in logarithmic units Logarithmic units of pitch are semitone and cents hundredths of a semitone A semitone is the interval the twelfth root of two There are twelve semitones per octave Each semitone is the pitch of a note of the chromatic scale An interval is determined by calculating a chromatic scale based on the tonic pitch and adding the number of semitones that correspond to the desired interval For example 4 semi tones are added for major third interval 7 semitones are added for a perfect fifth interval At all times whenever the student sings the student hears his voice on the headphones 15 and sees a tem poral visual representation of his voice s aural parame ters on the display screen 22 The control unit 12 con tains various switches 18 that when pressed perform the following functions PLAY 23 The audio tape 16 plays The student hears recorded material from audio tape 16 along with his voice on
24. dustries cover pp 26 27 and 43 real time pitch recognition algorithm for Music Applications 1980 Hayden Book Company Inc Rochelle park New Jersey Chapter 12 William B Kuhn Computer Journal vol 14 No 3 Fall 1990 pp 60 71 Primary Examiner Michael L Gellner Assistant Examiner P Stanzione 5 ABSTRACT An apparatus to assist the musical instruction of a stu dent including one or more tone sources e g flute human voice violin playing monophonically as inputs means to quantify aural parameters of pitch amplitude duration and timbre of inputs means to visually store and represent temporal history of said means to store and playback audio signal of inputs and means to arrive at and display musical related pitches A waveform analyzer is used to control a tunable low pass filter in the pitch tracker and preform spectral estimation of timbre Pre recorded music instruction and a karaoke player are used as examples of reference tone sources Visual representation of aural parameters can provided as video output 40 Claims 7 Drawing Sheets U S Patent Feb 22 1994 Sheet 1 of 7 5 287 789 Sheet 2 of 7 5 287 789 Feb 22 1994 U S Patent e 9t 0 oianv TWLISIG YAZAIWNV W3O3 3AVM YSHOLIMS 430023 JNOL QNVWWOD SSHILIMS TANVd 1NO34 U S Patent Feb 22 1994 Sheet 3 of 7 5 287 789
25. e and musical instruction Other sources of musical reference and instruction may be used without deviating from the spirit of the inven tion A perspective view of the invention 10 can be seen as set forth in FIG 1 The four main components are a headset 11 a control unit 12 a host computer 13 and a video monitor 41 The headset 11 contains a micro phone 14 to pick up student intonations and headphones 15 for the student to monitor his voice and recorded material The control unit 12 contains an instructional audio tape 16 a stereo audio tape player 17 various control switches 18 a connector 19 for the headset 11 and a serial communication output 20 The control unit 12 sends aural parameters to the host computer 13 over the serial communication line 21 The host computer 13 receives aural parameters and generates a video signal 40 that contains a visual representation of the aural parameters The video signal 40 is sent to a video moni tor 41 and viewed on the display screen 22 The host computer 13 is used as a display driver for the video monitor 41 to decrease the cost of the invention 10 for someone who already has a computer e g an IBM PC or Apple MacIntosh However it is not required and both the host computer 13 and video monitor 41 may be replaced with a portable display e g a liquid crystal and appropriate control electronics to make a compact battery powered portable version of the invention The headphones 15 pr
26. ecifically pre pared for his invention This requires the production marketing and distribution of music exercise data Pitch tracking is the dynamic determination of the fundamental frequency of an applied audio signal Much work has been done developing the art of pitch tracking for speech recognition Niedzwiecki and Mikiel 1976 Hess Wolfgang Pitch Determination of Speech Signals Volume 3 of Scringer Series in Informa tion Sciences Springer Verlag New York page 175 5 287 789 3 report of a pitch tracker using a tunable low pass filter whose cutoff is dynamically adjusted by the amplitude of the output signal If a signal is present at the output the cutoff frequency is lowered until the amplitude of the output goes down Ideally the adaptive operation of this system would dynamically maintain the cutoff fre quency of the filter slightly above the fundamental fre quency of the applied audio signal In addition to the reported problem of tracking performance being depen dent on input signal level it has been found through experiment that the output signal may produce noisy tracking results due to its small signal to noise ratio The systems of Mercer and Tumblin rely on pitch trackers that require one and only one peak per pitch cycle and an amplitude envelope that does not fluctuate rapidly For example when upper harmonics over tones of a resonant low pitched male voice are rein forced as they fall within t
27. ecified by the LOOP IN MARKER and LOOP OUT MARKER The loop in time specifies where in the monophonic reference note event list the looping segment begins and the loop out time specifies where the looping segment ends When the LOOP switch 27 is pressed the loop out time is set to the system clock The display period is the time it takes aural pa rameter data to scroll completely across the screen The loop in time is calculated as the loop out time minus the display period The loop in and loop out times are sent to the host computer by the LOOP IN MARKER and LOOP OUT MARKER commands respectively These times may be adjusted to allow a particular section of the looping segment to be repeated The loop in and loop out times may be independently and continuously adjusted for example by two slide potentiometers not shown read by two of the MCU 50 analog to digital aural parameter source pitch amplitude time stamp TABLE 1 Command Description Data DRAW TONE Plots aural parameters LOOP Activates loop function CLEAR SCREEN Clears the screen SHOW STAFF Draws musical staff SET SPEED Sets speed of display SHOW MAZE Draws maze SHOW INTERVAL Plots interval bars SHOW HARMONY Plots harmony bar LOOP IN Loop begin time time stamp MARKER LOOP OUT Loop end time time stamp MARKER aural parameter source tonic aural parameter source toni Several commands specify the aural parameter source since the invention can analyze and display aura
28. ed by the command tone decoder 53 to set the multiplexer 57 through the control line 62 to select the accompaniment signal 63 as the source to be digi tized The multiplexer 57 may also be set manually for example by switch not shown There are cases when the multiplexer 57 would be set to digitize the accompa niment signal 63 such as when the monophonic refer ence signal 55 contains the reference aural parameters but not the audible tone reference In one embodiment the monophonic reference source is a recording of tones of steady pitch and constant timbre and amplitude e g from a synthesizer which produces correspondingly consistent visual representations used as a model of an idealized singer The monophonic reference pitches would sound irritating i e machine like due to their aural consistency so an audible tone reference of a per son singing is provided on the accompaniment channel The two channels are synchronized so that each tone occurring on the monophonic reference channel corre sponds to a tone on the accompaniment channel The microphone 14 converts the voice of the student into an electric signal 71 that is amplified by an audio pre amplifier 64 and applied to a waveform analyzer 65 and the audio mixer 59 The waveform analyzer 65 is identical waveform analyzer 56 The gain of the audio pre amplifier 64 may be adjusted by the student not shown to compensate for variations in microphone 14 sensitivities
29. ematic of FIG 5 and the waveforms of FIG 6C when the applied waveform 306 is below the ground reference 308 the zero cross detector s output 205 is low which resets the droop detector s dual D type flip flops 216 and 217 e g National Semiconduc tor 74L S74 Upon reset the output 251 Q1 of flip flop 216 is low and the inverting output 232 Q2 of flip flop 217 is high The nature of a D type flip flop s out put is to adopt the state of its D input when clocked When the waveform 107 falls below the threshold VLOW 212 the rising output 214 of comparator 210 clocks a high state into flip flop 216 and a low state into flip flop 217 If the waveform 107 falls below the threshold VLOW 212 for a second time in the same pitch cycle as it does at 310 a high is clocked into flip flop 217 producing a low on the inverting output Q2 232 which is saved by the latch 271 When an input to the flip flop 271 goes low e g DROOP 232 the output stays high DROOP 237 until the MCU 50 reads the line and resets the flip flop 271 by pulsing the reset line 236 low The fol lowing discussion on filter control will use the positive logic version of signals i e DROOP 237 as compared to the negative logic version DROOP 232 If the GATE 240 is low there is no applied input signal 55 present filter control routine does not change the filter cutoff frequency If the GATE 240 is HIGH the filter 5 20 25 30 35 40
30. epresenting a sequence of pitch values 37 The apparatus of claim 35 wherein said mono phonic reference source is stored on said compact disk as a note list 38 The apparatus of claim 35 wherein the computer means includes means for displaying the pitch of student intonations as a line whose vertical position increases with the pitch of the student intonations 39 The apparatus of claim 35 wherein the monopho nic reference comprise solo singing 40 The apparatus of claim 39 further including means for digitally storing in the computer means a history of the sequence of pitches of the monophonic reference source for replay on the visual display means
31. er 56 and 65 respec tively and a headset 11 In case 5 the monophonic refer ence source is a note event list of aural parameters digi tally stored on a floppy disk Since the monophonic reference source is inaudible note events are sent by the MCU 50 or host computer 13 to a synthesizer not shown which provides the audible tone reference When note events are used as the monophonic refer ence source the displayed aural parameters of the monophonic reference source can lead lag or be in unison to the audible tone reference Typically it is desirable to have the visual display of reference aural parameters lead the audible tone reference so the stu dent can look ahead to upcoming notes For the case of a digitally stored note event list case 5 of Table 2 one TABLE 2 STORAGE MONOPHONIC WAVEFORM AUDIBLE TONE CONFIG MEDIA REFERENCE ANALYZER REFERENCE 1 audio tape note events R NO solo band L 2 audio tape solo vocal R YES solo R band L 3 audio tape pure tone R YES solo band L 4 none teacher YES teacher 5 floppy note event list NO synthesizer disk Nomenclature right channel of audio tape source L left channel of audio tape source All these configurations use a waveform analyzer 65 to extract aural parameters from student intonations The heading CONFIG refers to the configuration em bodiment The heading STORAGE MEDIA refers to an example of media on which information e g monopho
32. event sound from the audible tonal reference and accompaniment source from reach ing the microphone 14 which would otherwise add additional frequencies and noise that could result in pitch tracking errors Extraneous noise from the envi 5 20 25 30 35 40 45 50 55 60 65 8 Tonment be reduced by using directional type microphone The stereo audio tape 16 contains two channel a the accompaniment channel which contains verbal instruc tions for the student and musical accompaniment e g a piano and b the monophonic reference channel which contains intonations of the vocal instructor and encoded commands to control functions of the invention In order to understand some of the functions of the invention some musical terms must be introduced A tone is an audible sound that has pitch timbre duration and amplitude An interval is the frequency relationship between two pitches A scale is a series of pitches that are related by specific intervals A note is any pitch of a scale The tonic is the first note of a scale A harmony is two notes played together A melody is a sequence of notes A harmonic melody is a sequence of harmonies A common scale in Western music is the major scale which contains 7 pitches A chromatic scale is a se quence of 12 pitches based on the interval the twelfth root of two A pitch that is not equal to any note of a scale is called an accidental If the pitch is not exa
33. f the droop detector 105 and gate detector 106 and is read and reset by the MCU 50 An envelope follower 109 tracks the amplitude of the applied audio signal 55 and is read by an ADC input 110 of the MCU 50 A spectral estimator 11 detects changes in the slope of the waveform of the applied audio signal 55 The output of the spectral estimator 111 is counted by a pulse accumulator input 114 PAI of the MCU 50 Different configurations of the invention 10 may be formed based on the selection of storage technique monophonic reference source and audible tone refer ence Several embodiments are presented as set forth in Table 2 below 5 10 15 20 25 14 tracted from the solo vocal performance and the stu dent monitors both the solo vocal and accompanying band for a tonal reference The configuration of case 3 records a pure tone with steady pitch constant timbre and amplitude on the right channel and a combination audio mix of solo vocal and accompanying band on the left channel This configuration produces a visual representation of an ideal performance The pure tone is typically annoying to listen to due to its lusterless con sistent nature so the left channel provides a mix of solo vocal and band as a tonal guide to the student Case 4 uses a live musical instructor playing along with the student For example a singing teacher would sing along with the vocal student The teacher and student each require a waveform analyz
34. he frequency range of for mants the natural resonance frequencies of the vocal track multiple peaks can occur A musically trained listener can detect pitch errors as small as 0 3 a deviation of about one cycle per second for an A4 440 Hz The accuracy and stability of a pitch tracker is therefore very important in a music training system Timbre refers to tonal quality of a musical instru ment the combinations of overtones that give each instrument its unique sound The quality of a voice and the scratchy sound of a violin are all refer ences to timbre Fourier analysis is one technique to quantify timbre by measuring the energy in the compo nent frequencies of a sound source The analysis how ever requires numerous computations and is time con suming Nickerson et al use a bank of 19 filters to deter mine spectral content of the deaf student s voice An analog electronic implementation of such a filter bank would require many parts that occupy circuit board space undesirable in a portable unit and would have an impact on manufacturing time and cost digital imple mentation would require signal processing capabilities with associated speed requirements and cost Both ap proaches produce an abundance of data that must be further processed in order to be interpreted A pre ferred analysis technique would require few compo nents have a low cost and produce results that are easy to interpret
35. he functions of the individual system components may be integrated to produce an economic and contained 5 287 789 21 system taking advantage of similar operations such as image processing in the karaoke player 503 and video mixing by the digital effects processor and mixer 507 Such integration of hardware can be done without de parting from the spirit of the invention It is conceivable that a karaoke machine can be constructed to provide as a built in feature a visual representation of aural parameters superimposing or replacing the pro grammed material as taught here to guide assist and instruct the singer While the above provides a full and complete disclo sure of the invention modifications integrations alter nate implementations and constructions will naturally occur to those skilled in the art without departing from the true spirit and scope of the invention Therefore the above description and illustrations should not be con strued as limiting the scope of the invention which is defined in the following claims claim 1 A music training apparatus for teaching a student correct pitch and other musical skills by reference to a stored or pre recorded musical sequence comprising display means for presenting a live image visible to the student means for receiving a monophonic reference source and sending a variable signal in accordance there with to the visual display means so that the display means
36. hing a table of frequencies for all audible semitones about 100 and linearly interpreting between them further gain in conversion speed can be achieved if the frequency difference between neighboring semitones is also stored in the table Ideally the frequency output of the zero cross detec tor 102 corresponds directly to the pitch of the applied audio signal 55 Noise on the input 107 to the zero cross comparator 204 and transients as the filter 103 cutoff frequency is changed can cause deviations from the ideal pitch value Pitch tracking errors can be reduced by filtering In preferred embodiment of the pitch tracker extreme values of pitch are eliminated the re maining samples are averaged In a preferred embodi ment the last 24 samples of pitch value are saved and 5 287 789 17 arranged in ascending order as entry 1 to 24 and middle eight values are averaged entry 9 to 16 In calculating the average further computational speed is realized by shifting the binary sum three places to the right rather than dividing by eight The number of samples used 24 in this example can be reduce for greater response time or increased to achieve greater filtering Gate Detector 106 The gate detector 106 indicates to the MCU 50 that the applied audio signal 55 is present The gate detector 106 is a comparator 208 whose output 209 GATE goes low when the applied audio signal 55 is greater than the voltage reference VGATE 2
37. ition of each bar 400 The constant amplitude is visually indicated by the uniform thickness of the bars 400 The constant timbre is visually indicated by the uniform color not shown of each bar 400 The aural parameters of the student are represented by the line 402 whose thickness 403 represents amplitude and disappearance 404 indicates silence Timbre is repre sented by the color of the line 402 not shown In FIG 8 a series of three monophonic reference notes 405 406 and 407 of constant timbre and amplitude and stable pitch are plotted along with the aural param eters of the student segments 408 415 delineated for purposes of illustration by dashed vertical lines not actually seen on the display screen 22 The display speed is fast enough to reveal features of the student s voice Segment 408 shows good pitch matching fol lowed by poor pitch sustaining ability in segment 409 Segment 410 shows vibrato that is centered below the reference tone 405 This would sound flat to a trained musical ear Segment 411 shows a late start with over shoot in segment 412 when approaching the desired tone 406 followed by poor correction in segment 413 5 10 15 20 25 30 35 40 45 Segment 414 shows undershoot followed by a fast accurate recovery in segment 415 FIG 9 shows the aural parameters of intonations of the human voice as the monophonic reference source 450 plotted with the aural parameters of intonati
38. l parameters of multiple sources e g student intonation and monophonic reference source The DRAW TONE command contains a time stamp which specifies the absolute time of aural parameter occurrence The host computer 13 uses the time stamp to locate the position X axis of aural parameters on the display 50 screen 22 The time stamp is derived from a system time 55 clock maintained by the MCU 50 The aural parameter duration is not sent in the DRAW TONE command for it is apparent in the temporal nature of the display screen 22 i e the duration of an aural parameter is seen as its horizontal length The SHOW INTERVAL and 60 SHOW HARMONY commands specify the tonic of the scale to be used for calculating interval and harmony pitches The LOOP function plays out a digital recording of sampled audio visually represented on the display screen 22 In one preferred embodiment the last several seconds of audio material are saved by the MCU 50 in RAM 67 and the most recent segment of monophonic 65 ports or adjusted in increments of monophonic refer ence note events by push buttons A note event list is a complete record of every note with its associated aural parameters occurring in a piece of music One feature of the invention is the display of aural parameters of notes in advance of student intona tion Looking ahead in a musical score at upcoming notes allows a musician to phrase a series of notes build up volume
39. ll amplitude of the applied wave form 55 The envelope follower 109 extracts the maxi mum amplitude of the waveform with the peak tracking circuit composed of diode D13 and capacitor C13 Re sistor R19 discharges capacitor C13 to allow the enve 5 287 789 19 lope follower 109 to track waveforms of decreasing amplitude Operational amplifier 233 e g National Semiconductor LM324 is configured as a voltage fol lower to buffer the capacitor C13 from the analog to digital input 110 of the MCU 50 If an AGC automatic gain control circuit is used anywhere in the signal path of the applied waveform 55 for example to compensate for variations in microphone 14 output it is preferred to have the envelope follower 109 receive the applied waveform 55 before the AGC to assure a valid measurement of waveform amplitude Referring to FIGS 7 to 12 several examples of visual representations of aural parameters used in the pre ferred embodiment of the invention are shown In all of the visual representations of FIG 7 to 12 the vertical axis represents pitch and the horizontal axis represents time In FIG 7 a sequence of three notes from a monopho nic reference source of constant timbre and amplitude and stable pitch such as produced by a square wave generator appears as discrete horizontal bars 400 paral lel horizontal reference lines 401 as shown in the draw ing figures The stable pitch is visually indicated by the fixed vertical pos
40. mporal visual record of aural parameters of their musical performance and of a musical reference SUMMARY OF THE INVENTION principal object of the invention is to provide train ing assistance and guidance to music students in the development of musical skills Another object of the invention is to allow music students to visually monitor temporal representations of musical parameters of their musical performance and a musical reference A further object of the invention is to be compatible with a wide range of musical reference sources Another object of the invention is to provide a pitch tracker capable of determining the pitch of a wide range of monophonic audio inputs Another object of the invention is to provide a simple and inexpensive means to measure the timbre of an audio signal Another object of the invention is to provide a means to practice intonating intervals assisted by a temporal history of visual representations of aural parameters further object of the invention is to provide a means to practice intonating melodies assisted by a temporal history of visual representations of aural pa rameters In order to accomplish these objectives the invention comprises a means to provide a monophonic reference source a means to objectively and accurately quantify the aural parameters of pitch duration amplitude and timbre of student intonations and that of the monopho nic reference source a means for digitally
41. nel known to one skilled in the art such as frequency and amplitude modulation The pre 10 ferred embodiment uses the presence and absence of a fixed high frequency command tone e g 10 kHz to encode a standard asynchronous serial data format e g Electronic Industries Association interface standard RS232 C Referring to FIG 4 the composite audio 15 signal 52 is applied to the high pass filter 115 that atten uates all frequencies lower than the command tone The filtered signal 116 is applied to a tone decoder 117 whose output 118 goes low when the command tone is present The output 118 of the tone decoder 117 is ap 20 plied to the serial receive input RCV 119 of the MCU 50 A preferred embodiment of the tone decoder 117 uses an 1 567 e g National Semiconductor 2900 Semi conductor Drive Santa Clara Calif 95051 described 25 in the Linear Data Book 1988 edition Volume 3 page 5 62 to 5 67 A preferred embodiment of the high pass filter 115 is a two pole high pass filter not shown taught on page 175 of Don Lancaster s book Active Fil ter Cookbook Howard W Sams amp Co Inc 4300 West 30 62nd St Indianapolis Ind 46268 Many functions features and data can be encoded in the tone commands The tone commands can be viewed as a way of providing additional programming for the MCU 50 It is possible to load operating code for the 35 MCU 50 over the reference channel using tone encoded commands Some of the uses of the t
42. nic and audible tone reference can be stored Audio tape can be replaced with any media capable of storing two channels of audio e g L P compact disk D A T laser disk Floppy disk can be replaced with any media that can store digital information e g RAM ROM optical disk hard disk The heading MONO PHONIC REFERENCE refers to the source of the monophonic reference The symbols R and L refer to the right and left channel of the audio tape These assignments are arbitrary and are used to indicate chan nel allocation The heading WAVEFORM ANALY ZER indicates whether the configuration requires a second waveform analyzer 56 to extract aural parame ters from the monophonic reference The heading AU DIBLE TONE REFERENCE indicates the source of the audible tones the student hears as a tonal guide In case 1 tone encoded note events are stored on the right channel of an audio tape An audio mix of solo vocal and accompanying band is provided on the left channel to guide the student Case 2 may be used with commercially available music recorded in the multi plexed format Reference aural parameters are ex 45 50 55 60 65 Software implementation would have two pointers reading out of the note event list One pointer would select note events to be displayed the other pointer would select note events to send to the synthesizer The offset of these two pointers would determine how far ahead the display was from synthesi
43. oice Typically students would practice sing ing with the reference solo voice and accompaniment After they have learned the words and are comfortable singing the melody they turn off the reference solo voice and sing unassisted with the accompaniment More elaborate karaoke systems use a laser disk or CD amp G compact disk a format that encodes graphic images with audio that display song lyrics on a video monitor which change color as each word is sung anal ogous to the bouncing ball technique Karaoke sys tems do not evaluate the singer s performance and hence students must rely on their own musical percep tions for guidance Electronic devices exist which visually indicate the instantaneous absolute pitch and error of a tone source e g Sabine ST 1000 Chromatic Auto Tuner Korg DT 2 Digital Tuner Arion HU 8400 Chromatic Tuner Mercer U S Pat No 4 273 023 discloses a de vice that displays the instantaneous absolute pitch of a musical instrument with an array of LEDs arranged on a musical staff but can only display the pitch of one tone source at a time Tumblin U S Pat No 4 321 853 dis closes a system that measures the instantaneous pitch of a musical instrument relative to an electronically gener ated reference tone and displays the difference the pitch error on a column of lights Neither of these systems provide a time history of pitch nor do they provide any quantative indication of timbre or ampli tude
44. one encoded com mands are turn functions on and off e g loop interval harmony set the tonic display characters and text on display screen 22 e g words to songs instructions 40 point to items on the display screen 22 e g aural pa rameters of the monophonic reference source provide note event list provide lyrics and display reference aural parameters in advance of hearing the audible tone reference 45 Referring now to FIG 5 the detailed operation of each section the waveform analyzer 56 will be exam ined Spectral Estimator 111 The timbre of the applied audio signal 55 is estimated and quantified by counting 50 the number of times the slope of the waveform changes sign ripples These ripples generally arises from harmonics in the signal and are observed to increase per pitch cycle with the energy in the upper harmonics The spectral estimator 111 counts the number of ripples 55 in the waveform The audio signal 55 to be analyzed is applied to the non inverting input 200 of the comparator 203 The output 201 of the comparator 203 is low pass filtered by the combination of the resistor R10 and capacitor C10 60 and applied to the inverting input 202 of the comparator 203 The inverting input 202 lags the applied audio signal 55 and the output 201 changes each time the applied audio input 55 makes an appreciable change in slope sign and magnitude The number of changes in the 65 output 201 of the compara
45. ons of the student 451 The sources of intonation aural parame ters 450 and 451 are visually distinguished from each other by brightness indicated by the dashed line 451 and solid line 450 The overall vertical displacement of the student s tone line 451 below the monophonic refer ence line 450 indicates that the student is consistently Singing flat FIG 10 illustrates a monophonic reference source 420 of constant pitch timbre and amplitude plotted with the aural parameters of student intonation 421 along with a visual representation of the musical inter vals third 422 fifth 423 and eighth 424 octave Inter vals 422 424 are distinguished from reference tone 420 and student tone 421 by brightness indicated by dashed line FIG 11 illustrates three notes 425 427 of a melody played by a monophonic reference source of constant timbre amplitude and stable pitch Bars 428 430 indi 50 55 60 65 20 cate harmony notes e g a third interval above the three monophonic reference notes 425 427 Visualiza tion of student aural parameters are represented by line 431 Harmony bars 428 430 are distinguished from monophonic reference notes 425 427 and student into nation 431 by brightness indicated by dashed line FIG 12 illustrates a maze where the boundary 465 represents the pitch range the singer must confine their voice to in order to navigate the maze The student s aural parameters are plotted left to right and p
46. ophonic reference source The visual represen tation of aural parameters 520 provided by the inven tion 10 are visually superimposed with the visual image 521 provided by the karaoke player 503 The karaoke player 503 displays a visual image of the lyrics 521 provides a solo lead vocal audio channel 505 the monophonic reference source and an accompany ing music audio channel 506 the audible tone reference to a song selected from the optical disk 504 Referring to FIG 2 the karaoke player 503 replaces the tape player 17 as the monophonic and audible tone refer ence Audio channel 505 and 50 is applied to channel switcher 51 through audio lines 47 and 48 respectively Two video sources may be electronically combined to produce one image that contains the visual material of one source superimposed onto the other This tech nique is commonly used to add titles to video as in a video character generator For purposes of illustration a digital effects processor and mixer 507 Panasonic WIJAVES is used to combine the video signal 40 from the invention 10 with the video signal 531 from the karaoke player 503 The combined video signal 532 is displayed on a video monitor 41 The karaoke player 503 contains a combination of digital control analog signal and image processing hardware necessary to carry out the numerous tasks of the player 503 For clarity the components of the appli cation shown in FIG 14 are kept discrete In practice t
47. or maintaining the cutoff frequency of the filter means substantially near the frequency of the audio input signal 30 The apparatus of claim 29 wherein the filter means includes a tunable lowpass filter 31 The apparatus of claim 30 wherein the cutoff frequency of the filter is decreased when the filter out put crosses a fixed amplitude threshold more than two times per pulse of said pulse train 32 The apparatus of claim 31 wherein the cutoff frequency of the filter is increased when the output of the filter fails to cross a fixed lower amplitude thresh old 5 287 789 10 20 25 30 35 40 45 50 55 60 65 24 33 The apparatus of claim 31 wherein cutoff frequency of the filter is decreased when the reference waveform crosses a fixed upper amplitude threshold 34 The apparatus of claim 32 wherein the fixed am plitude threshold is the lower threshold 35 A music training apparatus for teaching a student correct pitch and other musical skills by reference to a stored or pre recorded musical sequence comprising audible speaker means for presenting audio to the student visual display means for presenting live images to the student a compact disk including audio and visual information a compact disk player for reading the information stored on the compact disk providing audio to the audible speaker means and visual images means for receiving a monophonic reference source from the
48. play format used to display said temporal history of the series of intonations made by the student 9 The apparatus of claim 1 further including a wave form analyzer which includes the pitch tracker means and further includes an amplitude detection means for providing information on the display means relating to the student s amplitude as well as meter and pitch 10 The apparatus of claim 9 wherein the computer means includes means for displaying pitch as height of a line segment on the display means and amplitude as thickness of the line on the display means 11 The apparatus of claim 1 further including a waveform analyzer which includes said pitch tracker means and further includes a spectral estimator for pro viding information on the display means relating to the student s timbre as well as meter and pitch 12 The apparatus of claim 11 wherein the computer means further includes means for displaying pitch as height of a line segment on the display means and tim bre as color of the line on the display means 13 The apparatus of claim 12 wherein said spectral estimator comprises means responsive to the change in slope of a waveform applied to said pitch tracker and means to count the number of changes in slope of the waveform 14 The apparatus of claim 13 wherein the change in slope detecting means includes a comparator means having first and second inputs and an output capaci tor connected between the second input
49. rogress through the maze is indicated by the line 466 The per formance of the student is displayed in the form of a score 467 calculated by the host computer 13 The score 467 is incremented as the student maintains their pitch within the boundary 465 of the maze e g at location 468 and subtracted each time the student s pitch ex ceeds the boundary 465 of the maze e g location 469 difficulty of the maze is determined by the width 470 of the boundary 465 the display speed and the sharpness of the boundary turns in the maze e g cor ner at location 471 This exercise helps increase the vocal range and pitch control of the student in the en joyable context of a game Further applications and embodiments of the inven tion are derived from the use of the video output 40 provided by the invention 10 FIG 13 shows the invention 10 used with a video tape recorder 500 The video output 40 of the host computer 13 is applied to the video input 510 of the video tape recorder 500 The combined audio signal 46 from the control unit 12 is applied in parallel to both the student s headset 11 and the audio input 511 of the video tape recorder 500 through an audio connector 501 commonly referred to as connector The video record can be used for example by a music instructor to review the homework of a student FIG 14 shows an application of the invention 10 using a karaoke player 503 e g the JVC RC GX7 as the mon
50. shows a sequence of pitch values the mono phonic reference source comprising desired pitches to be achieved by the student means for providing an audible tone reference to the student in unison with the monophonic reference source and sharing the same fundamental sequence of tones as the monophonic reference source pickup means for picking up intonations made by the student in attempting to follow the audible tone reference l tunable filter means to attenuate harmonics of a signal from the pickup means filter control means responsive to the droop of the output of the filter means for setting the frequency of the filter means substantially near the frequency of the signal from the pickup means pitch tracker means receiving a signal from the tun able filter means for analyzing and tracking the pitch of the student s intonations and for generating a pitch track signal and computer means for receiving pitch track signals from the pitch tracker means and for generating a signal to the display means to display visually a temporal history of the series of intonations made by the student superimposed with the desired se quence of pitch values as represented by the mono phonic reference source and shown on the display 2 The apparatus of claim 1 wherein the monophonic reference source comprises an audible tone and includ ing a second pitch tracker means receiving the mono phonic reference source the second pitch tracker means
51. stored as note event lists are commercially available e g Music Productions Inc P O Box 22681 Milwaukee Oreg 97222 0681 or Pass port 625 Miramontes Street Half Moon Bay Calif 94019 Commercial note event lists are commonly ar 5 287 789 5 ranged as orchestrated pieces of music with several parts tracks occurring simultaneously Such note event lists are used as monophonic reference sources by designating one track as the melody and the rest as accompaniment This designation can either be ascer tained from the manufacture or discovered by trial Once designated the host computer 13 can retain this information to relieve the student of any further con cern over track designation The means of reading a note event list with a computer and sending note event commands to a synthesizer have become greatly simpli fied with the adoption of the MIDI Musical Instrument Digital Interface standard is well known to those skilled in the art and need not be covered here Student intonations are monophonic tones produced or caused by the actions of the student including sing ing and playing a musical instrument Student intona tions are received and converted by a pickup device such as a microphone or electromagnet into an electric signal applied to the invention The pitch of a monophonic tone source is determined by a pitch tracker In a preferred embodiment of the invention the upper harmonics of a tone source are
52. the human voice violin and flute and excludes most percussive instruments e g snare drum and tam bourine tone has aural parameters that include pitch amplitude duration and timbre When used in the context of audible tone reference tone can include any combination of pitched and unpitched sound sources e g a band with a percussion section A basic ability required of a student of music is to produce and sustain a musical tone of defined pitch and good timbre This task is easy on an instrument like a piano which mechanically quantizes pitch and con strains timbre Singers however must dynamically adjust their vocal muscles to control pitch and timbre based on their aural perceptions Similarly violinists must adjust their bowing and fingering based o their aural perceptions The importance of these aural perceptions is demon strated in the difficulty deaf children have learning to speak If the internal discernment of pitch and timbre is not developed in an individual some external feedback is necessary In their paper titled Computer Aided Speech Training for the Deaf Journal of Speech and Hearing Disorders February 1976 Vol 41 No 1 R S Nickerson D N Kalikow and K N Steven report on 5 10 20 25 30 35 a computer based system that uses visual displays of 40 speech parameters e g pitch amplitude and spec trum to aid speech training for the deaf In music instruction a st
53. the time horizontal scale of the display screen 22 A fast display speed re veals details of pitch trajectory for example vibrato depth and speed pitch stability and reference pitch approach A slower display allows more pitches to be displayed on the display screen 22 SWITCH CHANNEL 31 This switch reverses the output of the stereo tape player 17 This function allows either channel of a stereo audio tape 16 to be treated as the monophonic reference source REFERENCE VOLUME 32 This control varies the volume of the monophonic reference channel heard on the headphones 15 ACCOMPANIMENT VOLUME 33 This control varies the volume of the accompaniment channel of the tape player 17 heard on the headphones 15 MIC VOLUME 339 This control varies the volume of the student s voice heard on the headphones 15 MASTER VOLUME 35 This control varies the combined volume of all audio signals applied to the headphones 15 The particular intervals and harmony or alternative scales e g pentatonic or other non Western scales can be preset selected by encoded tone on the monophonic reference channel or manually set by additional switches not shown In some cases it might be difficult to set the tonic as described as in the case of pre recorded songs The tonic may also be set by manual switches not shown For example two switches could respectively increment and decrement the tonic in semi tone increments and the selected tonic e g C
54. tor 203 is counted by the pulse accumulator input PAT 114 Pulses on the PAI 114 input increments an internal counter inside the 16 MCU 50 The single PAI 114 is shared by the two waveform analyzers 56 and 65 through the multiplexer 113 Referring to FIG 6A and 6B waveform 300 and 301 represent the audio signal 55 applied to the spectral estimator 111 of the vocalized sound and respectively These two sounds have different timbre sound has less energy in the upper harmonics and is perceived as having a more of a pure tonal qual ity as compared to the ee sound which has more of a nasal sound In some styles of singing e g choral the former is preferred Waveform 302 and 303 represent the low passed version occurring at the inverting input 202 of the comparator 203 Waveform 304 and 305 represent the output of the comparator 203 The vowel contains more energy in upper harmonics than the vowel as seen by the greater numbers of ripples appearing in waveform 301 as compared to waveform 300 This is seen by the greater number of transitions occurring in waveform 305 as compared to waveform 304 A C Coupled Half Wave Rectifier 125 Referring to FIG the post filtered audio signal 230 is a c coupled by capacitor 11 to remove any bias half wave rectified by diode D10 and attenuated by the resistor divider of R11 and R12 to reduce the maximum
55. udent s aural perceptions are typically developed through collaboration with a music teacher who points out by verbal comment and audible example the pitch timbral and timing errors of the student Teaching musical skills are complicated by the fact that sound unlike paintings cannot directly be seen and only exist when played Audio tape recorders allow students to review their performances but do not pro vide any analysis A system of entertainment that offers learn by exam ple instruction is the karaoke system popularized in Japan A karaoke system literally Japanese for hollow orchestra consists of a pre recorded audio source a microphone audio mixer amplifier and speaker The audio source material typically a compact or laser disk LaserKaraok Pioneer LDCA Inc 2265 East 22th Street Long Beach Calif 90810 is specially prepared with musical accompaniment on one channel and a solo vocal reference on the other The musical accompani ment can be any musical instruments that provide tonal support for the singer The accompaniment is usually a band or orchestra but could simply be a piano other vocalists or a guitar The reference channel is typically the solo voice of a trained singer or a solo instrument like a clarinet or monophonic synthesizer The karaoke system allows the singer to independently adjust the 45 50 55 60 65 2 volume of his voice the accompaniment and the refer ence solo v
56. udio inputs to the audio mixer 59 The MCU 50 continually reads and controls the waveform analyzers 56 and 65 The MCU 50 processes the output of the waveform analyzers 56 and 65 and sends quantized values of aural parameters to the host computer 13 through display commands and data sent over a serial communication line 21 The host computer 13 runs a display program that converts display com mands and data into visual representations and outputs a video signal 40 to the display screen 22 A partial list of representative display commands with description and data if any are set forth in Table 1 below 5 10 15 20 25 30 12 reference aural parameters are stored in the host com puter 13 The LOOP command instructs the host com puter 13 to maintain the temporal display of the most recent segment looping segment of monophonic reference aural parameters and update student aural parameters while the MCU 50 plays back the last sev eral seconds of audio material The result is the student sees and hears an unchanging segment of the most re cent history of the monophonic reference source re peated endlessly along with an updating display sweep of his intonations The time duration of the looping segment looping time is dependent on the display speed the faster the display speed the shorter the looping time The looping time and the corresponding monophonic note events included in the looping seg ment is sp
57. with pitch plotted logarithmic units along the vertical axis and time in linear units along the horizontal axis Each source of aural parameters is represented by a line of unique brightness whose thickness represents amplitude and color represents timbre The means to record and playback audio signals digi tally are well known to those skilled in the art see Hal Chamberlin Musical Applications of Microprocessors 1980 Hayden Book Company Inc Rochelle Park N J chapter 12 In one embodiment the tone source is low pass filtered to prevent aliasing and sampled at regular intervals with an analog to digital converter ADC The digital samples are stored in RAM and played back through a digital to analog converter DAC followed by a low pass filter to remove sampling artifacts 0 20 N 5 35 45 50 60 65 6 The audible tone reference provides guide to assist students in their musical performance and may be a sequence of singular tones monophonic or a plural ity of tones polyphonic Sources of audible tone reference include live and recorded human voices mu sical instruments and electronic tone generators The audible tone reference can be as simple as a solo human voice or as complex as a full orchestra The monopho nic reference source may serve as the audible tone refer ence If the monophonic reference is inaudible as with a note event list a means such as an electronic
58. zer intonations For the case of note events tone encoded on an audio tape case 1 of Table 2 one implementation would have the audio tape prepared with the tone encoded note events leading the audible ton events The host computer 13 could be informed of the magnitude of the offset through a tone encoded command When a note event list is used as the source of aural parameters and the audible tone reference does not occur on the accompaniment channel some other source of audible tone reference is needed As previ ously mentioned this could be a synthesizer or a com puter based tone generator The MCU 50 may also be used as a tone source For example the MCU 50 can generate tones by reading a wavetable stored in mem ory and send samples out to a DAC see Musical Appli cations of Microprocessors chapter 13 A more basic tone generator is available in the MCU 50 by configur ing the internal timer to generate a periodic rectangular wave The duty cycle of the rectangular wave may be varied to alter the perceived volume and timbre 5 287 789 15 The block diagram of electronic circuitry set forth in FIG 4 details the command tone decoder 53 which detects the presence of a command tone on the applied audio signal 52 The monophonic reference signal 52 is a composite 5 audio signal that contains a monophonic reference source and tone encoded commands for the MCU 50 There are several techniques of communicating data over an audio chan

Download Pdf Manuals

Related Search

Related Contents

Whirlpool ADO15 Dehumidifier User Manual  Fine blasting unit Cemat-NT 4/3/2 - Wassermann Dental    説明書 - GANZ  NEC E552 Mechanical Drawings  Philips 32PFL7482 Flat Panel Television User Manual  Northstar 6kw Open Array Radar System  LA LETTRE AUX LYCéEns  

Copyright © All rights reserved. Failed to retrieve file