【鸿蒙开发】如何实现语音搜索功能?
实现语音搜索功能:通过配置权限、音频采集器、语音识别
·
分为三大步骤:配置权限、音频采集器、语音识别,实现这三大板块,就实现了语音搜索功能,如果想要增加额外效果,可以加一个按压振动、动画效果。
写在前面:
配置权限需要用到:配置module.json5、鉴权checkAccessToken()、申请权限requestPermissionsFromUser()、二次申请权限requestPermissionOnSetting()
语音识别:speechRecognizer
音频采集器:AudioCapturer的API说明
开发步骤:
1. 鉴权并向用户申请权限:
(1)在module.json5中配置麦克风权限(使用场景,使用原因)
{
"name": "ohos.permission.MICROPHONE",
"usedScene": {
'abilities': [
'PhoneAbility'
]
},
"reason": "$string:app_name"
},
(2)通过调用checkAccessToken()方法来校验当前是否已经授权
import { abilityAccessCtrl, bundleManager, Permissions } from '@kit.AbilityKit';
import { BusinessError } from '@kit.BasicServicesKit';
async function checkPermissionGrant(permission: Permissions): Promise<abilityAccessCtrl.GrantStatus> {
let atManager: abilityAccessCtrl.AtManager = abilityAccessCtrl.createAtManager();
let grantStatus: abilityAccessCtrl.GrantStatus = abilityAccessCtrl.GrantStatus.PERMISSION_DENIED;
// 获取应用程序的accessTokenID
let tokenId: number = 0;
try {
let bundleInfo: bundleManager.BundleInfo = await bundleManager.getBundleInfoForSelf(bundleManager.BundleFlag.GET_BUNDLE_INFO_WITH_APPLICATION);
let appInfo: bundleManager.ApplicationInfo = bundleInfo.appInfo;
tokenId = appInfo.accessTokenId;
} catch (error) {
const err: BusinessError = error as BusinessError;
console.error(`Failed to get bundle info for self. Code is ${err.code}, message is ${err.message}`);
}
// 校验应用是否被授予权限
try {
grantStatus = await atManager.checkAccessToken(tokenId, permission);
} catch (error) {
const err: BusinessError = error as BusinessError;
console.error(`Failed to check access token. Code is ${err.code}, message is ${err.message}`);
}
return grantStatus;
}
async function checkPermissions(): Promise<void> {
let grantStatus1: boolean = await checkPermissionGrant('ohos.permission.LOCATION') === abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED;// 获取精确定位权限状态
let grantStatus2: boolean = await checkPermissionGrant('ohos.permission.APPROXIMATELY_LOCATION') === abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED;// 获取模糊定位权限状态
// 精确定位权限只能跟模糊定位权限一起申请,或者已经有模糊定位权限才能申请精确定位权限
if (grantStatus2 && !grantStatus1) {
// 申请精确定位权限
} else if (!grantStatus1 && !grantStatus2) {
// 申请模糊定位权限与精确定位权限或单独申请模糊定位权限
} else {
// 已经授权,可以继续访问目标操作
}
}
(3)动态向用户申请授权,通过调用requestPermissionsFromUser()方法来实现
import { abilityAccessCtrl, common, Permissions } from '@kit.AbilityKit';
import { BusinessError } from '@kit.BasicServicesKit';
const permissions: Array<Permissions> = ['ohos.permission.MICROPHONE'];
// 使用UIExtensionAbility:将common.UIAbilityContext 替换为common.UIExtensionContext
function reqPermissionsFromUser(permissions: Array<Permissions>, context: common.UIAbilityContext): void {
let atManager: abilityAccessCtrl.AtManager = abilityAccessCtrl.createAtManager();
// requestPermissionsFromUser会判断权限的授权状态来决定是否唤起弹窗
atManager.requestPermissionsFromUser(context, permissions).then((data) => {
let grantStatus: Array<number> = data.authResults;
let length: number = grantStatus.length;
for (let i = 0; i < length; i++) {
if (grantStatus[i] === 0) {
// 用户授权,可以继续访问目标操作
} else {
// 用户拒绝授权,提示用户必须授权才能访问当前页面的功能,并引导用户到系统设置中打开相应的权限
return;
}
}
// 授权成功
}).catch((err: BusinessError) => {
console.error(`Failed to request permissions from user. Code is ${err.code}, message is ${err.message}`);
})
}
@Entry
@Component
struct Index {
aboutToAppear() {
// 使用UIExtensionAbility:将common.UIAbilityContext 替换为common.UIExtensionContext
const context: common.UIAbilityContext = getContext(this) as common.UIAbilityContext;
reqPermissionsFromUser(permissions, context);
}
build() {
// ...
}
}
(4)当应用通过requestPermissionsFromUser()拉起弹框请求用户授权时,用户拒绝授权。应用将无法再次通过requestPermissionsFromUser拉起弹框,需要通过调用requestPermissionOnSetting(),直接拉起权限设置弹框,引导用户授予权限
import { abilityAccessCtrl, Context, common } from '@kit.AbilityKit';
import { BusinessError } from '@kit.BasicServicesKit';
let atManager: abilityAccessCtrl.AtManager = abilityAccessCtrl.createAtManager();
let context: Context = getContext(this) as common.UIAbilityContext;
atManager.requestPermissionOnSetting(context, ['ohos.permission.MICROPHONE']).then((data: Array<abilityAccessCtrl.GrantStatus>) => {
console.info('data:' + JSON.stringify(data));
}).catch((err: BusinessError) => {
console.error('data:' + JSON.stringify(err));
});
2. 初始化语音识别器:
(1)创建引擎
import { speechRecognizer } from '@kit.CoreSpeechKit';
import { BusinessError } from '@kit.BasicServicesKit';
let asrEngine: speechRecognizer.SpeechRecognitionEngine;
let sessionId: string = '123456';
// 创建引擎,通过callback形式返回
// 设置创建引擎参数
let extraParam: Record<string, Object> = {"locate": "CN", "recognizerMode": "short"};
let initParamsInfo: speechRecognizer.CreateEngineParams = {
language: 'zh-CN',
online: 1,
extraParams: extraParam
};
// 调用createEngine方法
speechRecognizer.createEngine(initParamsInfo, (err: BusinessError, speechRecognitionEngine: speechRecognizer.SpeechRecognitionEngine) => {
if (!err) {
console.info('Succeeded in creating engine.');
// 接收创建引擎的实例
asrEngine = speechRecognitionEngine;
} else {
console.error(`Failed to create engine. Code: ${err.code}, message: ${err.message}.`);
}
});
(2)设置监听项
// 创建回调对象
let setListener: speechRecognizer.RecognitionListener = {
// 开始识别成功回调
onStart(sessionId: string, eventMessage: string) {
console.info(`onStart, sessionId: ${sessionId} eventMessage: ${eventMessage}`);
},
// 事件回调
onEvent(sessionId: string, eventCode: number, eventMessage: string) {
console.info(`onEvent, sessionId: ${sessionId} eventCode: ${eventCode} eventMessage: ${eventMessage}`);
},
// 识别结果回调,包括中间结果和最终结果
onResult(sessionId: string, result: speechRecognizer.SpeechRecognitionResult) {
console.info(`onResult, sessionId: ${sessionId} sessionId: ${JSON.stringify(result)}`);
},
// 识别完成回调
onComplete(sessionId: string, eventMessage: string) {
console.info(`onComplete, sessionId: ${sessionId} eventMessage: ${eventMessage}`);
},
// 错误回调,错误码通过本方法返回
// 如:返回错误码1002200006,识别引擎正忙,引擎正在识别中
// 更多错误码请参考错误码参考
onError(sessionId: string, errorCode: number, errorMessage: string) {
console.error(`onError, sessionId: ${sessionId} errorCode: ${errorCode} errorMessage: ${errorMessage}`);
}
}
// 设置回调
asrEngine.setListener(setListener);
(3)开始识别
// 开始识别
private startListeningForWriteAudio() {
// 设置开始识别的相关参数
let recognizerParams: speechRecognizer.StartParams = {
sessionId: this.sessionId,
audioInfo: { audioType: 'pcm', sampleRate: 16000, soundChannel: 1, sampleBit: 16 } //audioInfo参数配置请参考AudioInfo
}
// 调用开始识别方法
asrEngine.startListening(recognizerParams);
};
private startListeningForRecording() {
let audioParam: speechRecognizer.AudioInfo = { audioType: 'pcm', sampleRate: 16000, soundChannel: 1, sampleBit: 16 }
let extraParam: Record<string, Object> = {
"recognitionMode": 0,
"vadBegin": 2000,
"vadEnd": 3000,
"maxAudioDuration": 20000
}
let recognizerParams: speechRecognizer.StartParams = {
sessionId: this.sessionId,
audioInfo: audioParam,
extraParams: extraParam
}
console.info('startListening start');
asrEngine.startListening(recognizerParams);
};
传入text,并存储到状态变量。
3. 开启语音采集器:
(1)初始化后进入回调函数,配置音频采集参数并创建AudioCapturer实例(AudioCapturerOptions)
import { audio } from '@kit.AudioKit';
let audioStreamInfo: audio.AudioStreamInfo = {
samplingRate: audio.AudioSamplingRate.SAMPLE_RATE_48000, // 采样率
channels: audio.AudioChannel.CHANNEL_2, // 通道
sampleFormat: audio.AudioSampleFormat.SAMPLE_FORMAT_S16LE, // 采样格式
encodingType: audio.AudioEncodingType.ENCODING_TYPE_RAW // 编码格式
};
let audioCapturerInfo: audio.AudioCapturerInfo = {
source: audio.SourceType.SOURCE_TYPE_MIC,
capturerFlags: 0
};
let audioCapturerOptions: audio.AudioCapturerOptions = {
streamInfo: audioStreamInfo,
capturerInfo: audioCapturerInfo
};
audio.createAudioCapturer(audioCapturerOptions, (err, data) => {
if (err) {
console.error(`Invoke createAudioCapturer failed, code is ${err.code}, message is ${err.message}`);
} else {
console.info('Invoke createAudioCapturer succeeded.');
let audioCapturer = data;
}
});
(2)调用on('readData')方法,订阅监听音频数据读入回调。
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo as fs } from '@kit.CoreFileKit';
class Options {
offset?: number;
length?: number;
}
let bufferSize: number = 0;
let path = getContext().cacheDir;
let filePath = path + '/StarWars10s-2C-48000-4SW.pcm';
let file: fs.File = fs.openSync(filePath, fs.OpenMode.READ_WRITE | fs.OpenMode.CREATE);
let readDataCallback = (buffer: ArrayBuffer) => {
let options: Options = {
offset: bufferSize,
length: buffer.byteLength
}
fs.writeSync(file.fd, buffer, options);
bufferSize += buffer.byteLength;
};
audioCapturer.on('readData', readDataCallback);
(3)开始录制音频
import { BusinessError } from '@kit.BasicServicesKit';
audioCapturer.start((err: BusinessError) => {
if (err) {
console.error(`Capturer start failed, code is ${err.code}, message is ${err.message}`);
} else {
console.info('Capturer start success.');
}
});
(4)调用stop()方法停止录制
import { BusinessError } from '@kit.BasicServicesKit';
audioCapturer.stop((err: BusinessError) => {
if (err) {
console.error(`Capturer stop failed, code is ${err.code}, message is ${err.message}`);
} else {
console.info('Capturer stopped.');
}
});
(5)调用release()方法销毁实例,释放资源
import { BusinessError } from '@kit.BasicServicesKit';
audioCapturer.release((err: BusinessError) => {
if (err) {
console.error(`capturer release failed, code is ${err.code}, message is ${err.message}`);
} else {
console.info('capturer released.');
}
});
以下是音频录制的完整代码:
import { audio } from '@kit.AudioKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo as fs } from '@kit.CoreFileKit';
const TAG = 'AudioCapturerDemo';
class Options {
offset?: number;
length?: number;
}
let bufferSize: number = 0;
let audioCapturer: audio.AudioCapturer | undefined = undefined;
let audioStreamInfo: audio.AudioStreamInfo = {
samplingRate: audio.AudioSamplingRate.SAMPLE_RATE_48000, // 采样率
channels: audio.AudioChannel.CHANNEL_2, // 通道
sampleFormat: audio.AudioSampleFormat.SAMPLE_FORMAT_S16LE, // 采样格式
encodingType: audio.AudioEncodingType.ENCODING_TYPE_RAW // 编码格式
};
let audioCapturerInfo: audio.AudioCapturerInfo = {
source: audio.SourceType.SOURCE_TYPE_MIC, // 音源类型
capturerFlags: 0 // 音频采集器标志
};
let audioCapturerOptions: audio.AudioCapturerOptions = {
streamInfo: audioStreamInfo,
capturerInfo: audioCapturerInfo
};
let path = getContext().cacheDir;
let filePath = path + '/StarWars10s-2C-48000-4SW.pcm';
let file: fs.File = fs.openSync(filePath, fs.OpenMode.READ_WRITE | fs.OpenMode.CREATE);
let readDataCallback = (buffer: ArrayBuffer) => {
let options: Options = {
offset: bufferSize,
length: buffer.byteLength
}
fs.writeSync(file.fd, buffer, options);
bufferSize += buffer.byteLength;
};
// 初始化,创建实例,设置监听事件
function init() {
audio.createAudioCapturer(audioCapturerOptions, (err, capturer) => { // 创建AudioCapturer实例
if (err) {
console.error(`Invoke createAudioCapturer failed, code is ${err.code}, message is ${err.message}`);
return;
}
console.info(`${TAG}: create AudioCapturer success`);
audioCapturer = capturer;
if (audioCapturer !== undefined) {
(audioCapturer as audio.AudioCapturer).on('readData', readDataCallback);
}
});
}
// 开始一次音频采集
function start() {
if (audioCapturer !== undefined) {
let stateGroup = [audio.AudioState.STATE_PREPARED, audio.AudioState.STATE_PAUSED, audio.AudioState.STATE_STOPPED];
if (stateGroup.indexOf((audioCapturer as audio.AudioCapturer).state.valueOf()) === -1) { // 当且仅当状态为STATE_PREPARED、STATE_PAUSED和STATE_STOPPED之一时才能启动采集
console.error(`${TAG}: start failed`);
return;
}
// 启动采集
(audioCapturer as audio.AudioCapturer).start((err: BusinessError) => {
if (err) {
console.error('Capturer start failed.');
} else {
console.info('Capturer start success.');
}
});
}
}
// 停止采集
function stop() {
if (audioCapturer !== undefined) {
// 只有采集器状态为STATE_RUNNING或STATE_PAUSED的时候才可以停止
if ((audioCapturer as audio.AudioCapturer).state.valueOf() !== audio.AudioState.STATE_RUNNING && (audioCapturer as audio.AudioCapturer).state.valueOf() !== audio.AudioState.STATE_PAUSED) {
console.info('Capturer is not running or paused');
return;
}
//停止采集
(audioCapturer as audio.AudioCapturer).stop((err: BusinessError) => {
if (err) {
console.error('Capturer stop failed.');
} else {
fs.close(file);
console.info('Capturer stop success.');
}
});
}
}
// 销毁实例,释放资源
function release() {
if (audioCapturer !== undefined) {
// 采集器状态不是STATE_RELEASED或STATE_NEW状态,才能release
if ((audioCapturer as audio.AudioCapturer).state.valueOf() === audio.AudioState.STATE_RELEASED || (audioCapturer as audio.AudioCapturer).state.valueOf() === audio.AudioState.STATE_NEW) {
console.info('Capturer already released');
return;
}
//释放资源
(audioCapturer as audio.AudioCapturer).release((err: BusinessError) => {
if (err) {
console.error('Capturer release failed.');
} else {
console.info('Capturer release success.');
}
});
}
}
4. 最后,传入ArrayBuffer二进制音频数据识别成文字
let uint8Array: Uint8Array = new Uint8Array();
// 可以通过如下方式获取音频流:1、通过录音获取音频流;2、从音频文件中读取音频流
// 2、从音频文件中读取音频流:demo参考
// 写入音频流,音频流长度仅支持640或1280
asrEngine.writeAudio(sessionId, uint8Array);
5.额外内容
(1)添加一个振动效果,增加逼格
// 按住语音的时候 加一个马达震动
vibrator.startVibration({
type: 'time',
duration: 200,
}, {
id: 0,
usage: 'media'
})
停止振动
// 停止振动
vibrator.stopVibration(vibrator.VibratorStopMode.VIBRATOR_STOP_MODE_TIME)
(2)添加一个按压按键拉伸的效果
animateTo({ duration: 500 }, () => {
this.scaleValue = 1.5
})
手指抬起,恢复原大小
animateTo({ duration: 500 }, () => {
this.scaleValue = 1
})
7. 把上述过程封装成工具类(鉴权工具、语音识别、音频采集),解耦代码,提高复用和维护性
最后整合一下即可
更多推荐
所有评论(0)