억양합성을 통한 영어교육 Synthesizing Exaggerated F0 and duration

억양합성을 통한 영어교육Synthesizing Exaggerated F0 and duration

경남대학교 영어학부 윤규철

나사렛 대학교 교양교직학부 초청강연교양영어 추계학술행사

2007. 11. 16.

2

특강 내용

1. 영어의 운율… 3 쪽

2. 강세와 강조 … 8 쪽

3. 과장을 통한 억양연습 … 13 쪽

4. 과장된 억양의 합성 … 16 쪽

5. 과장된 길이의 합성… 28 쪽

6. 영어교육에의 응용 … 30 쪽부록 : 프랏 스크립트 … 33 쪽

1. 영어의 운율

English Prosody

4

1. 영어의 운율

• 운율이란 ?– 음의 높낮이 + 음의 길이 + 음의 강도

5

1. 영어의 운율

• 말소리– 분절음 ( 자 , 모음 ) + 초분절음 ( 운율 )

가나다라… +

6

1. 영어의 운율

• 운율 없는 말소리가 가능할까 ?

음의 높낮이 , 음의 길이 , 음의 강도가 정해지지 않고 소리가 날 수 있을까 ?

가로 , 세로 , 높이를 정하지 않고 건물을 만들어 내라는 것과 같은 소리…

7

1. 영어의 운율

• 음계의 ‘파’음을 악보에 그리지 않고 낼 수 있을까 ?

‘ 도레미파솔라시도’는 머리 속에만 있는 추상적 소리일 뿐…

2. 강세와 강조

Stress & Prominence

9

2. 강세와 강조

• 강세 (stress) 는 운율을 조작하여 강조함으로써 만든다– 음의 높낮이를 이용– 음의 길이를 이용– 음의 강도를 이용

예 ) banana [bənǽnə] A: “What did you have for lunch?” B: “A banana.” A: “A banana?”

10

2. 강세와 강조

• 강조는 주변보다 두드러짐을 나타낸다– 즉 , 주변환경 ( 주변음절들 ) 에 따라

정해진다– 말문장에 있어 , 특정 음절을 강조하려면

주변의 음절들보다 두드러지도록 만든다• 예 ) 주변의 음절들이 조용하면

강조음절을 시끄럽게• 예 ) 주변의 음절들이 시끄러우면

강조음절을 조용하게

11

2. 강세와 강조

12

2. 강세와 강조

다음 중 강조된 낱말은 ?

강조 1

강조 2

강조 3

3. 과장을 통한 억양연습

Exaggerating Intonation

14


• 종이를 90 도 각도로 접으려면 ?

(1) 89 도로 접는다(2) 90 도로 접는다(3) 90 도 이상으로 접었다가 조금 펴서 90 도로 만든다

15


• 영어 억양을 잘 익히려면 ?

조금 과장되게 연습한다

4. 과장된 억양의 합성

Synthesizing Exaggerated Intonation

17


• MOMEL algorithm(Hirst & Espesser, 1991) 을 이용한 극대점 / 극소점 위치 파악

• Praat implementation by Cyril Auranhttp://stl.recherche.univ-lille3.fr/sitespersonnels/auran/english/index.html

• 혹은 PSOLA algorithm(Moulines & Charpentier, 1990)을 이용하여 , 극대점 / 극소점을 수작업이나 혹은 프랏스크립트를 이용하여 ‘과장’하여 합성한다– MOMEL algorithm 을 이용하지 않고 프랏 Manipulation

object 의 “ Stylize pitch (2 st)” 명령을 이용할 수도 있다

http://stl.recherche.univ-lille3.fr/sitespersonnels/auran/english/index.html

18


19


20


21


22


23


24


25


26


27


5. 과장된 길이의 합성

Synthesizing lengthened duration

29

5. 과장된 길이의 합성

PSOLA algorithm(Moulines & Charpentier, 1990) 을 이용 , 극대점을 프랏스크립트를 이용하여 늘린다

6. 영어교육에의 응용

Implications for English Teaching

31

6. 영어교육에의 응용

• 주어진 영어문장을 자동으로 과장하여 학생들이 따라 말하고 들을 수 있도록 한다

• 억양합성 시스템을 구축하여 영어말하기 교육에 이용한다

• 과장하여 연습한 학생들과 그렇지 않은 학생들을 비교하여 그 차이를 살펴본다– 원어민을 이용한 평가의 경우 억양 이외의 운율 요소를

동일하게 조작하여 억양만을 판별할 수도 있다

감사합니다

################################################################## exaggerateIntonation.praat (Written by Kyuchul Yoon, [email protected])# Given a sound file, this script turns it into a Manipulation object, # stylizes it, and asks the user to manually 'exaggerate' the pitch contour to the# user's needs. When all done, it resynthesizes the sound and saves it as a# sound file in the input folder.#---------------------------------------------------------------------------# When exaggerating the pitch contour, the user has only to place the cursor# to a particular position by just clicking once and press CONTINUE button.# The user repeats this process until the desired pitch contour is created.# When the process is finished, the user DOES NOT move the cursor (letting # the script know that it's time to get out of the 'exaggerating' loop) and# press the CONTINUE button.#################################################################

form Parametersword inputFile_(wav) testnn.wavword inputFolder_(subfolder) nonnativeword outputFileSuffix_(to_be_appended_to_inputFile) -manipnatural numPoints_(per_parabola_for_resynthesis) 8comment Clean objects?choice CleanObject: 2

button Yesbutton No

endform

# Create the output file nameinputFilePrefix$ = inputFile$ - ".wav"outputFile$ = inputFilePrefix$ + outputFileSuffix$outputFile$ = outputFile$ + ".wav"

# Read the input file and make it a Manipulation objectRead from file... 'inputFolder$'/'inputFile$'Rename... inFileObjTo Manipulation... 0.01 75 600Rename... inFileManipObj

# 'Exaggerating' loopselect Manipulation inFileManipObjEditeditor Manipulation inFileManipObj

Stylize pitch (2 st)currentCursor = 0lastCursor = Get cursor

# Repeat this process until the user does not move the cursorwhile currentCursor <> lastCursor

lastCursor = currentCursorpause Place cursor at Max/Min. ... When all done, click CONTINUE without moving cursor.currentCursor = Get cursor# Remove the nearest existing pitch point and insert# a new 'exaggerated' pitch point to where the user# place the cursorRemove pitch point(s)Add pitch point at cursor

endwhile

# Make the pitch contour smoothInterpolate quadratically... numPointsPublish resynthesis

endeditor

Rename... outFileObjWrite to WAV file... 'inputFolder$'/'outputFile$'pause Check the input file folder for the resynthesized output file!

# Clean objects only when the user said "Yes"if cleanObject$ = "Yes"

select Manipulation inFileManipObjplus Sound inFileObjplus Sound outFileObjRemove

endif

A Praat script for manual exaggeration of intonation using stylization

##################################################################### exaggerateIntonation-editorScript.praat (Kyuchul Yoon, [email protected])# Run this script as an editor script from within the editor window of the# Manipulation object that is created during the execution of the # Momel_intsint-mono.praat script.# The user is expected to exaggerate the pitch contour point by point in the # Manipulation object window####################################################################

Stylize pitch (2 st)currentCursor = 0lastCursor = Get cursor

# Repeat this process until the user does not move the cursorwhile currentCursor <> lastCursor

lastCursor = currentCursorpause Place cursor at Max/Min. ... When all done, click CONTINUE without moving cursor.currentCursor = Get cursor# Remove the nearest existing pitch point and insert# a new 'exaggerated' pitch point to where the user# place the cursorRemove pitch point(s)Add pitch point at cursor

endwhile

# Make the pitch contour smoothInterpolate quadratically... 8Publish resynthesis

endeditorRename... exaggeratedpause Be sure to save 'exaggerate' Sound object into a WAV file!########## END OF EDITOR SCRIPT ##############

A Praat editor script for manual exaggeration of intonation of MOMEL Manipulation object

################################################################################# exaggerateIntonationAutomatically.praat (2007. 11.04 by Kyuchul Yoon, [email protected])# Given a sound file, this script stylizes the original pitch contour, and exaggerates it automatically.# Exaggeration algorithm:# 1. Compare two neighboring pitch points in Hz.# 2. The point with lower Hz gets boosted, while the lower one gets lowered. If same, nothing happens.# 3. Move to the next two points, the first of which is the second from the previous step. # (i.e. advances by one pitch point) Use values before boosting/lowering from the previous step.# 4. Repeat step 1. #################################################################################form Parameters

word inputFile_(wav) testnn.wavword inputFolder_(subfolder) nonnativeword outputFileSuffix_(to_be_appended_to_inputFile) -exaggeratednatural stylizeSemitones 3natural freqValue_(Hz_difference_for_boosting) 30natural numPoints_(per_parabola_for_resynthesis) 8

endform# Create the output file nameinputFilePrefix$ = inputFile$ - ".wav"outputFile$ = inputFilePrefix$ + outputFileSuffix$outputFile$ = outputFile$ + ".wav"# Read the input file and make it a Manipulation objectRead from file... 'inputFolder$'/'inputFile$'Rename... inFileObjTo Manipulation... 0.01 75 600Rename... manipOriginalCopy... inFileManipObj# Stylize the original sound file, leaving a copy of the stylized manipulation# object for later comparisonEditeditor Manipulation inFileManipObj

Stylize pitch... stylizeSemitones SemitonesClose

endeditorCopy... manipOriginalStylized# 'Exaggerating' loop. The exaggeration is performed on the pitch tier objectselect Manipulation inFileManipObjExtract pitch tierRename... inFilePitchTierObjnumPitchPoints = Get number of points# A flag to signal whether a pitch point was modified or notflagModified = 0

for i to numPitchPoints# For all the pitch points except for the last one. This is necessary because the 'exaggerating' # is done for two neighboring pitch points. The last one has no such neighbor.if i <> numPitchPoints

# Special treatment for the first pitch pointif flagModified = 0

iPrevHz = Get value at index... i# If it's not the first pitch point, the next pitch point from the earlier loop becomes# the previous pitch point in the current loopelse

iPrevHz = iNextHzendif# Get the Hz value and the time values of the two pointsiNextHz = Get value at index... (i+1)iPrevTime = Get time from index... iiNextTime = Get time from index... (i+1)# Decide which is biggerbiggerHz = imax(iPrevHz, iNextHz)# If the first of the two pitch points is bigger,then boost/lower it by freqValueif biggerHz = 1

flagModified = 1Remove point... iAdd point... iPrevTime (iPrevHz+freqValue)Remove point... (i+1)Add point... iNextTime (iNextHz-freqValue)

elsif biggerHz = 2flagModified = 1Remove point... iAdd point... iPrevTime (iPrevHz-freqValue)Remove point... (i+1)Add point... iNextTime (iNextHz+freqValue)

# If same, do nothingelse

# Do nothingflagModified = 0

endifendif

endfor# Switch the exaggerated pitch tier objectplus Manipulation inFileManipObjReplace pitch tierselect Manipulation inFileManipObjCopy... manipExaggeratedselect Manipulation inFileManipObjEditeditor Manipulation inFileManipObj

Interpolate quadratically... numPointsPublish resynthesisClose

endeditorRename... outFileObjWrite to WAV file... 'inputFolder$'/'outputFile$'

A Praat script for automatic exaggeration of intonation using stylization

############################################################################# exaggerateF0-duration-button.praat (2007. 11.08)# (Written by Kyuchul Yoon, [email protected])# Given a sound file, this script stylizes the original pitch contour,# and exaggerates its F0 contour and peak durations automatically.# NOTE) Users can check graphically how the original sound gets exaggerated# by following the pop-up Manipulation Editor window.# NOTE) Users can also select which parameter(s) to exaggerate:# (1) F0 contour only# (2) Duration only (for the F0 peaks)# (3) Both############################################################################# F0 exaggeration algorithm:# 1. Compare two neighboring pitch points in Hz.# 2. The point with higher Hz gets boosted, while the lower one gets lowered.# If same, nothing happens.# 3. Move to the next two points, the first of which is the second from the # previous step. (i.e. advances by one pitch point) # Use values before boosting/lowering from the previous step.# 4. Repeat step 1. ############################################################################## Duration exaggeration algorithm:# 1. Lengthen stylized pitch tier object peaks durValue times.# 2. Neighboring valleys maintain the original duration.############################################################################## Installation procedure:# (1) Start Praat : Praat > Open Praat script...# (2) Open this script "exaggerateF0-duration-button.praat# (3) In the script editor window : File > Add to dynamic menu...# (4) Replace "Do it..." with something like "Exaggerate!"# (5) Click [OK]# (6) Close Script Editor window# (7) You can now launch the script from within Praat using the "Exaggerate!"# button when a sound object is selected.############################################################################

form Parameterscomment F0 exaggeration by boosting peaks and lowering valleyschoice ExaggerateFrequency: 1

button Yesbutton No

natural stylizeSemitones 2natural freqValue_(Hz_diff_for_boosting_and_lowering) 30comment

=========================================================================comment Duration exaggeration by lengthening peakschoice ExaggerateDuration: 1

button Yesbutton No

real durValue_(times_orig_duration_for_lengthening) 1.5comment

=========================================================================comment For resynthesis (Default value would suffice)natural numFreqPoints_(per_parabola_for_resynthesis) 8

endform

To Manipulation... 0.01 75 600Rename... inFileManipObjCopy... originalEditselect Manipulation inFileManipObjEditeditor Manipulation inFileManipObj

Stylize pitch... stylizeSemitones SemitonesClose

endeditorCopy... original_stylized

select Manipulation inFileManipObjExtract pitch tierRename... inFilePitchTierObjnumPitchPoints = Get number of points# Check user's choiceif (exaggerateFrequency$ = "Yes" and exaggerateDuration$ = "Yes")

call procExaggerateFreq freqValuecall procExaggerateDur durValue

select Manipulation inFileManipObjRename... exaggerated_F0_DurEdit

elsif (exaggerateFrequency$ = "Yes" and exaggerateDuration$ = "No")call procExaggerateFreq freqValue

select Manipulation inFileManipObjRename... exaggerated_F0Edit

elsif (exaggerateFrequency$ = "No" and exaggerateDuration$ = "Yes")call procExaggerateDur durValue

select Manipulation inFileManipObjRename... exaggerated_DurEdit

endif############### END OF MAIN FUNCTION ###############

A Praat script for automatic exaggeration of F0 & durationusing stylization

procedure procExaggerateFreq fValueselect PitchTier inFilePitchTierObjflagModified = 0for i to numPitchPoints

# For all the pitch points except for the last one# This is necessary because the 'exaggerating' is done for two# neighboring pitch points. The last one has no such neighbor.if i <> numPitchPoints

# Special treatment for the first pitch pointif flagModified = 0

iPrevHz = Get value at index... ielse

iPrevHz = iNextHzendif# Get the Hz value and the time values of the two pointsiNextHz = Get value at index... (i+1)iPrevTime = Get time from index... iiNextTime = Get time from index... (i+1)

# Decide which is biggerbiggerHz = imax(iPrevHz, iNextHz)# If the first of the two pitch points is bigger,# then boost it by freqValue and lower the second by as muchif biggerHz = 1

flagModified = 1Remove point... iAdd point... iPrevTime (iPrevHz+fValue)Remove point... (i+1)Add point... iNextTime (iNextHz-fValue)

# If the second is bigger, then do similar jobselsif biggerHz = 2

flagModified = 1Remove point... iAdd point... iPrevTime (iPrevHz-fValue)Remove point... (i+1)Add point... iNextTime (iNextHz+fValue)

else# Do nothingflagModified = 0

endifendif

endfor# Switch the exaggerated pitch tier objectplus Manipulation inFileManipObjReplace pitch tierselect Manipulation inFileManipObjEditeditor Manipulation inFileManipObj

Interpolate quadratically... numFreqPointsPublish resynthesisClose

endeditorRename... exaggerated

endproc

procedure procExaggerateDur dValueselect PitchTier inFilePitchTierObj# Identify the pitch peaks# Initialize the index of the peak array variablek = 0# Except for the first and last pitch point, check if it's a peakfor j from 2 to (numPitchPoints-1)

leftPointFreq = Get value at index... (j-1)rightPointFreq = Get value at index... (j+1)currentPointFreq = Get value at index... jindexOfMax = imax(leftPointFreq, currentPointFreq, rightPointFreq)

# If the current point is at the peak, remember the time coordinate# in an array variable timeOfPeaks. Remember its neighboring valleysif indexOfMax = 2

k = k + 1timeOfPeaks'k' = Get time from index... jtimeOfLeftValley'k' = Get time from index... (j-1)timeOfRightValley'k' = Get time from index... (j+1)

endifendfor

select Manipulation inFileManipObjExtract duration tierRename... inFileDurTierObj

# Add duration points at the peak locationsfor m to k

peakTime = timeOfPeaks'm'leftPeakTime = timeOfLeftValley'm'rightPeakTime = timeOfRightValley'm'Add point... peakTime dValueAdd point... leftPeakTime 1Add point... rightPeakTime 1

endfor

# Switch the exaggerated duration tier objectplus Manipulation inFileManipObjReplace duration tierselect Manipulation inFileManipObjEditeditor Manipulation inFileManipObj

Interpolate quadratically... numFreqPointsPublish resynthesisClose

endeditorRename... exaggerated

endproc######## END OF SCRIPT ##########

억양합성을 통한 영어교육 Synthesizing Exaggerated F0 and duration

Documents