基于飞浆Paddle的Android字幕实时提取

介绍

本项目是给瞎子供给的一款看电影的实时英文字幕读取的软件;首要选用的技术：
MediaProjection截取屏幕 + AccessibilityService监听手势 + 开源O通信工程CR飞浆Paddle + 进程间通讯 + 讯飞TTS语音组成

一,对接PaddleLite编译相关文件

1,下载paddle官网的相关demo

官网文档
github.软件工程师com/PaddlePaddl…

项目地址
gitee.com/gewussj/Ocr…

2,装备AndroidStudio的NDK

• 把相关文件复制到自己的项目后，下载相关NDK，MappearanceARK用来编译咱们复制的文件
• 首要文件：
• 1 OpenCV
• 2 PaddleLite
• 将手势舞教程视频慢动作1和2文件放在项目app包之下
• 3 cpp
• 将3放在app/src/main您的包名下
• 4 assets
• 将4放在src/main之下 并设置为source root(选中文件右键Mark dir as)
• 5 jappleava文件包名ocr

3,下appointment载NDK和MARK

• 下载相关http://192.168.1.1登录匹配的NDK这儿就不多说了
• 装备您的build.grHTTPadle文件/http://192.168.1.1登录参阅官方demo的相同文件
• 修正Predhttp 404ictor.jaappetiteva文件的相关办法，使他适appreciate合您自己运用

二,新建无障碍服务http://www.baidu.com

• 我这儿首要完成了两个办法

    @Override
    public void onConfigurationChanged(Configuration newConfig) {
        super.onConfigurationChanged(newConfig);
        if (newConfig.orientation == 1) {
            ttsUtils.startSpeech("已进入竖屏,读屏现已暂停");
        } else if (newConfig.orientation == 2) {
            ttsUtils.startSpeech("已进入横屏,请运用手势开端读屏");
        }
        timerInstance.stopDefaultTimer();
    }
    @RequiresApi(api = Build.VERSION_CODES.N)
    @Override
    protected boolean onGesture(int gestureId) {
        if (gestureId == GESTURE_SWIPE_DOWN_AND_UP) {
            int h = ScreenUtils.isH(getApplicationContext());
            if (h == Surface.ROTATION_270 || h == Surface.ROTATION_90) {
                timerInstance.starDefaultTimer();
                ttsUtils.startSpeech("现已开端读屏");
            } else {
                ttsUtils.startSpeech("当时竖屏状况,请进入视频播映页面横屏下运用读屏");
                timerInstance.stopDefaultTimer();
            }
        }
        return true;
    }

• 第一个函数首要是实时监测手机横屏或许竖屏

• 第二个函数便是监测咱们手势舞的手势

  • 当咱们在屏幕上履行（先下后上手势）时此onGesture函数会履行，我这通信技术儿由于需求的原因需求检测反正屏，竖屏状况下不履行。

• 第二个函数如何装备履行

  • 1在无障碍描述http协议文件通信工程增加答应手势操作
  • android:手势舞视频accessibilityFlags=”flagRe通信行程卡下载appportViewIds|flagIncludeNotI手势舞视频mportantViews|flagR手势舞equestTouchExplorationMode|flagRequestFingerprin软件tGestures”
  • 这是我的悉数符号 由于我不只需求监测手势，还需求监测包名等其他的操作
  • 注意：

  • android:canRequestTouchExplorationMode = "true"
    android:canRequestFingerprintGestures="true"
    

  • 这两句将会非常重要

• 悉数如下

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackGeneric|feedbackSpoken|feedbackVisual"
    android:accessibilityFlags="flagReportViewIds|flagIncludeNotImportantViews|flagRequestTouchExplorationMode|flagRequestFingerprintGestures"
    android:canRetrieveWindowContent="true"
    android:canRequestFilterKeyEvents="true"
    android:canPerformGestures="true"
    android:notificationTimeout="100"
    android:canRequestTouchExplorationMode = "true"
    android:canRequestFingerprintGestures="true"
    tools:targetApi="o" />

三,前台服务的辅助服务appear类

public class MediaProjrctService extends Service implements ImageReader.OnImageAvailableListener {
    private VirtualDisplay virtualDisplay;
    private MediaProjection mediaProjection;
    private Predictor predictor;
    private TtsUtils ttsUtils;
    private Activity activity;
    private int densityDpi;
    private  Context context;
    public Handler handler = new Handler(Looper.getMainLooper()) {
        @Override
        public void handleMessage(@NonNull Message msg) {
            if (msg.what == 0) {
                startScreenCapture();
            }
        }
    };
    @Override
    public void onCreate() {
        context = getApplicationContext();
        super.onCreate();
        Notification notification = createForegroundNotification();
        int NOTIFICATION_ID = 1;
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
            startForeground(NOTIFICATION_ID, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION);
        } else {
            startForeground(NOTIFICATION_ID, notification);
        }
        registerReceivers();
    }
    @Override
    public void onDestroy() {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
            stopForeground(STOP_FOREGROUND_REMOVE);
        } else {
            stopForeground(true);
        }
        stopScreenCapture();
        ttsUtils.cancleSpeech();
        unregisterReceiver(timerRecivier);
        super.onDestroy();
    }
    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        Activity activity = AppUtils.getActivity();
        MediaProjection mediaProjection = AppUtils.getMediaProjection();
        if (activity != null && mediaProjection != null) {
            this.activity = activity;
            this.mediaProjection = mediaProjection;
            this.densityDpi = activity.getResources().getDisplayMetrics().densityDpi;
            predictor = Predictor.getInstace();
            predictor.init(activity, AppUtils.assetModelDirPath, AppUtils.assetlabelFilePath);
            ttsUtils = TtsUtils.initSpeech(activity);
        }
        return super.onStartCommand(intent, flags, startId);
    }
    private int screenW;
    private int screenH;
    private int cenScreenW;
    private Bitmap bitmap;
    public void startScreenCapture() {
        if (mediaProjection != null) {
            screenW = AppUtils.getScreenW(activity);
            screenH = AppUtils.getScreenH(activity);
            @SuppressLint("WrongConstant")
            ImageReader mImageReader = ImageReader.newInstance(screenW, screenH, 0x1, 1);
            mImageReader.setOnImageAvailableListener(MediaProjrctService.this, null);
            virtualDisplay = mediaProjection.createVirtualDisplay("ScreenCapture", screenW, screenH, densityDpi,
                    DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR, mImageReader.getSurface(), null, null);
        }
    }
    @Override
    public void onImageAvailable(ImageReader reader) {
        Image image = reader.acquireNextImage();
        if (image != null) {
            final Image.Plane[] planes = image.getPlanes();
            if (planes.length > 0) {
                ByteBuffer buffer = planes[0].getBuffer();
                //每个像素的距离
                int pixelStride = planes[0].getPixelStride();
                //总的距离
                int rowStride = planes[0].getRowStride();
                int rowPadding1 = rowStride - pixelStride * screenW;
                if (bitmap == null) {
                    bitmap = Bitmap.createBitmap((screenW + (rowPadding1 / pixelStride)), screenH, Bitmap.Config.ARGB_8888);
                }
                try {
                    bitmap.copyPixelsFromBuffer(buffer);
                    recognitionText(bitmap);
                } catch (Exception e) {
                    e.printStackTrace();
                }
                image.close();
                stopScreenCapture();
            }
        }
    }
    private int textH;
    private int textY;
    private String text = "";
    public void recognitionText(Bitmap bitmap) {
        Bitmap bitmapBottom = null;
        if (correctSuccess) {
            bitmapBottom = AppUtils.cropBitmapCustom(bitmap, 0, textY, bitmap.getWidth(), textH);
            predictor.setInputImage(bitmapBottom);
        } else {
            cenScreenW = AppUtils.getScreenW(activity) / 2;
            predictor.setInputImage(bitmap);
        }
        boolean runModel = predictor.runModel();
        if (runModel) {
            List<String> textResult = predictor.getTextResult();
            if (textResult != null && textResult.size() != 0) {
                List<Point> textPotion = predictor.getTextPotion();
                //核算ocr回来的一共几组数据
                int textBody = textPotion.size() / 4;
                //遍历每一组 规则：由于每一组四个坐标 所以每组之后乘每组的个数
                for (int i = 0; i < textBody; i++) {
                    //核算每组数据的中心坐标
                    int textCenter = (textPotion.get(i * 4).x + textPotion.get(i * 4 + 1).x) / 2;
                    //判别每组数据的中心坐标  是否在规定的中心区域内  由于有差错 所以设置的是屏幕中心的-10/+10
                    if ((cenScreenW - 100) < textCenter && textCenter < (cenScreenW + 100)) {
                        if (correctSuccess) {
                            if (AppUtils.isSmooth(textResult.get(i), text)) {
                                text = textResult.get(i);
                                ttsUtils.startSpeech(textResult.get(i));
                                if (bitmapBottom != null && isSaveImg) {
                                    AppUtils.saveBitmap(bitmapBottom, activity);
                                }
                            }
                        } else {
                            //correctNum 是表明多少次之后 标价字幕位置
                            if (AppUtils.isSmooth(textResult.get(i), text)) {
                                text = textResult.get(i);
                                ttsUtils.startSpeech(textResult.get(i));
                                if (AppUtils.correctNum >= 10) {
                                    AppUtils.correctSuccess = true;
                                    //假如区域不设置大一点 则会呈现 辨认不到文字的状况
                                    textY = textPotion.get(i * 4).y - 50;
                                    textH = textPotion.get(i * 4 + 2).y + 50 - textY;
                                } else {
                                    correctNum++;
                                }
                                if (bitmap != null && isSaveImg) {
                                    AppUtils.saveBitmap(bitmap, activity);
                                }
                            }
                        }
                    }
                }
            }
        }
    }
    @Nullable
    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }
    private void registerReceivers() {
        IntentFilter intentFilter = new IntentFilter();
        intentFilter.addAction("ssjAction");
        timerRecivier = new TimerRecivier();
        registerReceiver(timerRecivier, intentFilter);
    }
    public Timer timer;
    public TimerTask timerTask;
    public TimerRecivier timerRecivier;
    public void starDefaultTimer() {
        if (timer == null) {
            timer = new Timer();
        }
        if (timerTask == null){
            timerTask = new TimerTask() {
                @Override
                public void run() {
                    handler.sendEmptyMessage(0);
                }
            };
        }
        timer.schedule(timerTask,  1000 ,800);
    }
    public void stopDefaultTimer() {
        if (timer != null) {
            timer.purge();
            timer.cancel();
            timer = null;
        }
        if (timerTask != null) {
            timerTask.cancel();
            timerTask = null;
        }
    }
    /**
     * 这儿的必定要设置为virtualDisplay = null
     * 尽管他会每次运用完毕自动开释 可是你仍是需求手动开释
     * 否则导致 bitmap 花屏
     */
    public void stopScreenCapture() {
        if (virtualDisplay != null) {
            virtualDisplay.release();
            virtualDisplay = null;
        }
    }
     /**
      * 接纳无障碍服务的手势 翻开or封闭
      */
    public class TimerRecivier extends BroadcastReceiver {
        @Override
        public void onReceive(Context context, Intent intent) {
            boolean isAction = intent.getBooleanExtra("isAction", false);
            if (isAction) {
                starDefaultTimer();
            } else {
                stopDefaultTimer();
            }
        }
    }
     /**
      * 服务初始化时 告诉栏显现一个 告诉
      */
    private Notification createForegroundNotification() {
        NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
        String notificationChannelId = "notification_channel_id_screen_capture";
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
            String channelName = "ScreenCapture";
            int importance = NotificationManager.IMPORTANCE_NONE;
            NotificationChannel notificationChannel = new NotificationChannel(notificationChannelId, channelName, importance);
            notificationChannel.setDescription("点明字幕辨认");
            notificationChannel.enableLights(false);
            notificationChannel.enableVibration(false);
            if (notificationManager != null) {
                notificationManager.createNotificationChannel(notificationChannel);
            }
        }
        NotificationCompat.Builder builder = new NotificationCompat.Builder(this, notificationChannelId);
        builder.setSmallIcon(R.mipmap.dianming_ocr);
        builder.setContentTitle(AppUtils.getString(context,R.string.notificationTitle));
        builder.setContentTitle(AppUtils.getString(context,R.string.notificationText));
        builder.setWhen(System.currentTimeMillis());
        return builder.build();
    }

四,截屏的功用完成

• 截屏的相关接口

  • 谷歌给咱们供给了一个截屏和录屏的对外类，咱们能够运用这个类履行相关的操作
  • MediaProje通信ction

• MediaProjection的封装

  • 由于java基础薄弱，所以选用了单例模式进行封装
  • 首要对外开放的便是，开手势密码图案大全端截屏，和完毕截屏
  • IapplemageReader.new通信达Instance(screenW, screenH, 0x1, 1)
  • 首要是这个：这个首要是创建想手势识别要获取的屏幕大小，后边两个都相同不必多操通信行程卡心

      private int screenW;
      private int screenH;
      private String text = "";
      private int cenScreenW;
      private Bitmap bitmap;
      public void startScreenCapture() {
          if (mediaProjection!=null){
              screenW = AppUtils.getScreenW(activity);
              screenH = AppUtils.getScreenH(activity);
              @SuppressLint("WrongConstant") ImageReader mImageReader = ImageReader.newInstance(screenW, screenH, 0x1, 1);
              mImageReader.setOnImageAvailableListener(this, null);
              virtualDisplay = mediaProjection.createVirtualDisplay("ScreenCapture", screenW, screenH,
                      densityDpi, DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR, mImageReader.getSurface(), null, null);
          }
      }

• 截屏后的回掉

    private Bitmap bitmap;
    @Override
    public void onImageAvailable(ImageReader reader) {
        Image image = reader.acquireNextImage();
        if (image != null) {
            final Image.Plane[] planes = image.getPlanes();
            if (planes.length > 0) {
                ByteBuffer buffer = planes[0].getBuffer();
                //每个像素的距离
                int pixelStride = planes[0].getPixelStride();
                //总的距离
                int rowStride = planes[0].getRowStride();
                int rowPadding1 = rowStride - pixelStride * screenW;
                if (bitmap == null) {
                    bitmap = Bitmap.createBitmap((screenW + (rowPadding1 / pixelStride)), screenH, Bitmap.Config.ARGB_8888);
                }
                try {
                    bitmap.copyPixelsFromBuffer(buffer);
                    EventBus.getDefault().post(bitmap);
                } catch (Exception e) {
                    Log.e("TAG", "onImageAvailable: " + e);
                }
                image.close();
                reader.close();//这个居然忘记了  不然ImageReader一向有可用图画时将会一向调用
                stopScreenCapture();
            }
        }
    }

• 这儿截屏后的回掉必定通信地址要注意，否则手机发烫严峻通信行程卡下载app。
• 1 Image image = reader.acqhttpclientuireNextImage();
• 这儿为什么不进行直接运用，由于这儿只能够获取一次，这行代码的意思是，获取截屏后回来的Image
• 个人认为有点类似与OkHttp的Body回来也是只能够获取appointment一次
• 2 imag软件库e.close(); reader.cloapprovese();
• 这儿必定要记住封闭,否则你会发现当你履行完毕截屏的时候此回掉通信地址是写什么地址依然会回来图片。

五,文字辨认

public void recognitionText(Bitmap bitmap) {
           if (correctSuccess) {
               Bitmap bitmapBottom = AppUtils.cropBitmapCustom(bitmap, 0, textY, bitmap.getWidth(), textH);
               predictor.setInputImage(bitmapBottom);
               AppUtils.saveBitmap2file(bitmapBottom,activity,new Date().toString(),"/ssj/");
           } else {
               cenScreenW = AppUtils.getScreenW(activity) / 2;
               predictor.setInputImage(bitmap);
               AppUtils.saveBitmap2file(bitmap,activity,new Date().toString(),"/ssj/");
           }
           boolean runModel = predictor.runModel();
           if (runModel) {
               List<String> textResult = predictor.getTextResult();
               //既然有字幕 那么字幕的坐标必定不为null
               if (textResult != null && textResult.size() != 0) {
                   List<Point> textPotion = predictor.getTextPotion();
                   //核算ocr回来的一共几组数据
                   int textBody = textPotion.size() / 4;
                   //遍历每一组 规则：由于每一组四个坐标 所以每组之后乘每组的个数
                   for (int i = 0; i < textBody; i++) {
                       //核算每组数据的中心坐标
                       int textCenter = (textPotion.get(i * 4).x + textPotion.get(i * 4 + 1).x) / 2;
                       //判别每组数据的中心坐标  是否在规定的中心区域内  由于有差错 所以设置的是屏幕中心的-10/+10
                       if ((cenScreenW - 10) < textCenter && textCenter < (cenScreenW + 10)) {
                           if (correctSuccess) {
                               if (AppUtils.isSmooth(textResult.get(i), text)) {
                                   text = textResult.get(i);
                                   ttsUtils.startSpeech(textResult.get(i));
                                   Log.i("TAG", "run: " + text);
                               }
                           } else {
                               if (AppUtils.isSmooth(textResult.get(i), text)) {
                                   text = textResult.get(i);
                                   ttsUtils.startSpeech(textResult.get(i));
                                   if (correctNum >= 10) {
                                       correctSuccess = true;
                                       textY = textPotion.get(i * 4).y - 20;
                                       textH = textPotion.get(i * 4 + 2).y + 20 - textY;
                                   } else {
                                       correctNum++;
                                   }
                                   Log.i("TAG", "run: " + text);
                               }
                           }
                       }
                   }
               }
           }
       }       

• 这儿我写的很清软件楚了
• ScreenUtils.getScreenW(getApplicationConte软件库xt())
• 获取屏http协议幕宽度
• predictor.getTextResult()
• 获取OCR回来文字组，是个调集
• predictor.getTextPotion()
• 获取OCR回来的文字坐标，也是个Potion调集，每个potion是四个点的坐标。
• ttsUtil手势语s.startSpeech(软件开发textResult.get(i));
• 读取悉数回来文字
• TextUtils.isSmooth(textResult.get(i), newtextResult)
• 判别回来字符是否契合逻辑
• 由于是字幕 这儿我判别了是否剧中 居中通信地址是写什么地址是上半屏仍是下半屏
• 然后第2次截取的是符号的上半屏/下半屏节约辨认时刻https和http的区别由于全屏软件测试辨认每次的辨认时刻大于内软件置的计时器时刻

六,无障碍服务的手势

 @RequiresApi(api = Build.VERSION_CODES.N)
    @Override
    protected boolean onGesture(int gestureId) {
        boolean isAction;
        if (gestureId == GESTURE_SWIPE_DOWN_AND_UP) {
            int h = AppUtils.isH(getApplicationContext());
            if (h == Surface.ROTATION_270 || h == Surface.ROTATION_90) {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.screenReadingStart));
                isAction = true;
            } else {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.screenReadingTip));
                isAction = false;
            }
            sndTimerCast(isAction);
            //保存辨认图片
        } else if (gestureId == GESTURE_SWIPE_DOWN_AND_LEFT) {
            AppUtils.isSaveImg = !AppUtils.isSaveImg;
            if (AppUtils.isSaveImg) {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationLife));
            } else {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationNoLife));
            }
            sendCast();
            //过滤弹幕
        } else if (gestureId == GESTURE_SWIPE_DOWN_AND_RIGHT) {
            AppUtils.isDanMu = !AppUtils.isDanMu;
            if (AppUtils.isDanMu) {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationRight));
            } else {
                ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationNoRight));
            }
            sendCast();
        } else {
            ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationError));
        }
        return true;
    }
    /**
     * 告诉mainActivity 更改Checkout的选中
     */
    private void sendCast() {
        Intent intent = new Intent(this, MainActivity.MyGuangBo.class);
        intent.putExtra("img", AppUtils.isSaveImg);
        intent.putExtra("danmu", AppUtils.isDanMu);
        sendBroadcast(intent);
    }
   /**
     * 告诉MediaProjrctService的开端截屏和停止截屏
     * @param action
     */
    private void sndTimerCast(boolean action) {
        AppUtils.correctSuccess = false;
        AppUtils.correctNum = 0;
        Intent intent = new Intent();
        intent.setAction("ssjAction");
        intent.putExtra("isAction", action);
        sendBroadcast(intent);
    }

• 手势气氛三类
• 1：下滑后上滑 （开端履行辨认字幕）
• 2：下滑后左滑 （翻开保存的辨认图片）
• 3：下滑后右滑 （翻开过滤视频弹幕）

七,东西类

• 一切的东西都在这

public class AppUtils {
    /*ocr的asset加载文件*/
    public static final String assetModelDirPath = "models/ocr_v2_for_cpu";
    public static final String assetlabelFilePath = "labels/ppocr_keys_v1.txt";
    public static final int PERMISSION_CODE = 100;
    //是否保存辨认的图片或许辨认弹幕
    public static boolean isSaveImg = false;
    public static boolean isDanMu = false;
    public static boolean correctSuccess = false;
    public static int correctNum = 0;
    private static MediaProjection mediaProjection;
    @SuppressLint("StaticFieldLeak")
    private static Activity activity;
    public static MediaProjection getMediaProjection() {
        return mediaProjection;
    }
    public static void setMediaProjection(MediaProjection mediaProjection) {
        AppUtils.mediaProjection = mediaProjection;
    }
    public static Activity getActivity() {
        return activity;
    }
    public static void setActivity(Activity activity) {
        AppUtils.activity = activity;
    }
    public static String getString(Context context, int id){
        return context.getString(id);
    }
    /**
     * 判别是否敞开
     * @return
     */
    public static boolean isAccessibilitySettingsOn(Context context, String className) {
        if (context == null) {
            return false;
        }
        ActivityManager activityManager = (ActivityManager) context.getSystemService(Context.ACTIVITY_SERVICE);
        if (activityManager != null) {
            List<ActivityManager.RunningServiceInfo> runningServices =
                    activityManager.getRunningServices(100);// 获取正在运转的服务列表
            if (runningServices.size() < 0) {
                return false;
            }
            for (int i = 0; i < runningServices.size(); i++) {
                ComponentName service = runningServices.get(i).service;
                if (service.getClassName().equals(className)) {
                    return true;
                }
            }
        }
        return false;
    }
    /**
     * 跳转到无障碍设置页面
     */
    public static void goToSettingPage(Context context) {
        Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS);
        context.startActivity(intent);
    }
    public static void saveBitmap2file(Bitmap bmp, Context context, String num, String sdPath) {
        String savePath;
        String fileName = num + ".JPEG";
        if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
            savePath = Environment.getExternalStorageDirectory().getPath() + sdPath;
        } else {
            Toast.makeText(context, "保存失利！", Toast.LENGTH_SHORT).show();
            return ;
        }
        File filePic = new File(savePath + fileName);
        try {
            if (!filePic.exists()) {
                filePic.getParentFile().mkdirs();
                filePic.createNewFile();
            }
            FileOutputStream fos = new FileOutputStream(filePic);
            bmp.compress(Bitmap.CompressFormat.JPEG, 100, fos);
            fos.flush();
            fos.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        // 最后告诉图库更新
        context.sendBroadcast(new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE, Uri.parse("file://" + savePath+fileName)));
    }
    /**
     * 裁剪必定高度保留下面
     *
     * @param srcBitmap  需求裁剪的bitmap
     * @param needHeight 需求裁剪出来的高
     */
    public static Bitmap cropBitmapBottom(Bitmap srcBitmap, int needwith, int needHeight) {
        /*裁剪保留下部分的第一个像素的Y坐标*/
        int needY = srcBitmap.getHeight() - needHeight;
        /*裁剪关键步骤*/
        return Bitmap.createBitmap(srcBitmap, needwith, needY, srcBitmap.getWidth(), needHeight);
    }
    public static Bitmap cropBitmapCustom(Bitmap srcBitmap, int firstPixelX, int firstPixelY, int needWidth, int needHeight) {
        if(firstPixelX + needWidth > srcBitmap.getWidth()){
            needWidth = srcBitmap.getWidth() - firstPixelX;
        }
        if(firstPixelY + needHeight > srcBitmap.getHeight()){
            needHeight = srcBitmap.getHeight() - firstPixelY;
        }
        /**裁剪关键步骤*/
        Bitmap cropBitmap = Bitmap.createBitmap(srcBitmap, firstPixelX, firstPixelY, needWidth, needHeight);
        return cropBitmap;
    }
    /**
     * 获取的是屏幕
     *
     * @param context
     */
    private static DisplayMetrics getDisplayMetrics(Context context) {
        DisplayMetrics metrics = new DisplayMetrics();
        getDispaly(context).getMetrics(metrics);
        return metrics;
    }
    /**
     * 获取屏幕管理器
     *
     * @param context
     */
    private static Display getDispaly(Context context) {
        WindowManager systemService =
                (WindowManager) (context.getApplicationContext().getSystemService(Context.WINDOW_SERVICE));
        return systemService.getDefaultDisplay();
    }
    /**
     * 获取屏幕宽度
     *
     * @param context
     */
    public static int getScreenW(Context context) {
        return getDisplayMetrics(context).widthPixels;
    }
    /**
     * 获取屏幕高度
     * 不包括顶部状况栏和底部navigationBar的高度
     *
     * @param context
     */
    public static int getScreenH(Context context) {
        return getDisplayMetrics(context).heightPixels;
    }
    /**
     * 屏幕旋转角度
     * 假如屏幕旋转90或许270是判别为横屏
     */
    public static int isH(Context context) {
        return ((WindowManager) context.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();
    }
    /**
     * 获取字符串持平的个数
     *
     * @param s1
     * @param s2
     * @return
     */
    public static boolean getEquals(String s1, String s2) {
        if (isEmpty(s1) && isEmpty(s2)) {
            int percent = getPercent(s1, s2);
            return percent <= 50;
        }else {
            return false;
        }
    }
    private static int getPercent(String s1, String s2) {
        int num = 0;
        int length1 = s1.length();
        int length2 = s2.length();
        for (int i = 0; i < length1; i++) {
            for (int j = 0; j < length2; j++) {
                char c1 = s1.charAt(i);
                char c2 = s2.charAt(j);
                if (c1 == c2) {
                    num++;
                }
            }
        }
        NumberFormat numberFormat = NumberFormat.getInstance();
        numberFormat.setMaximumFractionDigits(0);
        return Integer.parseInt(numberFormat.format((float) num / (float) length1 * 100));
    }
    /**
     * 判别字符串是否为空
     * @param s
     * @return
     */
    public static boolean isEmpty(String s) {
        return s == null && s.equals("");
    }
    /**
     * 判别是否包括特别字符
     * @param str
     * @return
     */
    public static boolean isSpecialChar(String str) {
        String regEx = "[0o _`~@#$%^&*+=|{}''\[\]<>/@#￥%……&* ——+|{}【】‘；：’、]|n|r|t";
        Pattern p = Pattern.compile(regEx);
        Matcher m = p.matcher(str);
        return m.find();
    }
    /**
     * 表明开头是英文
     *
     * @param s
     * @return
     */
    public static boolean isFristAZ(String s) {
        String frist = s.substring(0, 1);
        boolean matches = frist.matches("[a-zA-Z]+");
        return matches;
    }
    /**
     * 表明开头是数字
     *
     * @param s
     * @return
     */
    public static boolean isFristNum(String s) {
        String frist = s.substring(0, 1);
        return frist.matches("[0-9]+");
    }
    /**
     * @param s1  ocr回来的数据
     * @param s2  表明与第2次ocr回来的数据 比照的 也便是第一次的ocr的数据
     * @return
     */
    public static boolean isSmooth(String s1, String s2) {
        if ( !isFristNum(s1)  && !isFristAZ(s1) &&  !getEquals(s1,s2)  && !isSpecialChar(s1)  && !s1.equals(s2)) {
            return true;
        }else {
            return false;
        }
    }
}

八,MainAc手势舞教学视频简单tivity的完成

public class MainActivity extends AppCompatActivity implements View.OnClickListener {
    private MediaProjectionManager systemService;
    private TtsUtils ttsUtils;
    @SuppressLint("StaticFieldLeak")
    private static Switch saveImg;
    @SuppressLint("StaticFieldLeak")
    private static Switch danmu;
    private final String[] permissions = new String[]{READ_EXTERNAL_STORAGE, WRITE_EXTERNAL_STORAGE};
    private Context context;
    /**
     * 自定义广播接纳 手势操作开关
     */
    public static class MyGuangBo extends BroadcastReceiver {
        @Override
        public void onReceive(Context context, Intent intent) {
            boolean isImg = intent.getBooleanExtra("img", false);
            boolean isDanMu = intent.getBooleanExtra("danmu", false);
            setCheckOut(isImg, isDanMu);
        }
    }
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        context = getApplicationContext();
        getPermission();
        initView();
    }
    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        getMenuInflater().inflate(R.menu.toolbar_menu, menu);
        return true;
    }
    @SuppressLint("NonConstantResourceId")
    @Override
    public boolean onOptionsItemSelected(@NonNull MenuItem item) {
        if (item.getItemId() == R.id.helper) {
            ttsUtils.startSpeech(AppUtils.getString(context, R.string.usehelpr));
            AlertDialog.Builder builder = new AlertDialog.Builder(this);
            builder.setTitle("协助");
            builder.setMessage(AppUtils.getString(context, R.string.usehelpr));
            builder.setPositiveButton("确认", null);
            builder.show();
        }
        return super.onOptionsItemSelected(item);
    }
    @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
    private void initView() {
        Button bu_runder1 = findViewById(R.id.bu_runder1);
        Button bu_runder2 = findViewById(R.id.bu_runder2);
        Toolbar toolbar = findViewById(R.id.toolbar);
        bu_runder1.setOnClickListener(this);
        bu_runder2.setOnClickListener(this);
        ttsUtils = TtsUtils.initSpeech(this);
        setSupportActionBar(toolbar);
        saveImg = (Switch) findViewById(R.id.save_img);
        danmu = (Switch) findViewById(R.id.danmu);
        saveImg.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
            @Override
            public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
                AppUtils.isSaveImg = isChecked;
                if (getPermission()) {
                    ttsUtils.startSpeech("暂时没有获取权限");
                    return;
                }
                if (isChecked) {
                    ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationLife));
                } else {
                    ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationNoLife));
                }
            }
        });
        danmu.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
            @Override
            public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
                AppUtils.isDanMu = isChecked;
                if (isChecked) {
                    ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationRight));
                } else {
                    ttsUtils.startSpeech(AppUtils.getString(context,R.string.notificationNoRight));
                }
            }
        });
    }
    @Override
    protected void onResume() {
        setCheckOut(AppUtils.isSaveImg, AppUtils.isDanMu);
        super.onResume();
    }
    /**
     * 设置开关的选中状况
     */
    private static void setCheckOut(boolean isImg, boolean isDanMu) {
        if (saveImg == null || danmu == null) {
            return;
        }
        saveImg.setChecked(isImg);
        danmu.setChecked(isDanMu);
    }
    /**
     * 查看无障碍服务是否敞开
     */
    @SuppressLint("NonConstantResourceId")
    @Override
    public void onClick(View view) {
        switch (view.getId()) {
            case R.id.bu_runder1:
                if (systemService == null) {
                    initProject();
                } else {
                    ttsUtils.startSpeech(AppUtils.getString(context,R.string.screenCatOnInit));
                }
                break;
            case R.id.bu_runder2:
                initAccess();
                break;
        }
    }
    /**
     * 获取截屏服务
     */
    public void initProject() {
        systemService = (MediaProjectionManager) getSystemService(MEDIA_PROJECTION_SERVICE);
        Intent screenCaptureIntent = systemService.createScreenCaptureIntent();
        startActivityForResult(screenCaptureIntent, AppUtils.PERMISSION_CODE);
    }
    /**
     * 无障碍判别是否现已翻开
     */
    private void initAccess() {
        boolean accessibilitySettingsOn = AppUtils.isAccessibilitySettingsOn(MainActivity.this, AccessibilityServices.class.getName());
        if (!accessibilitySettingsOn) {
            AppUtils.goToSettingPage(MainActivity.this);
        }
    }
    @Override
    protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
        super.onActivityResult(requestCode, resultCode, data);
        if (requestCode != AppUtils.PERMISSION_CODE) {
            return;
        }
        if (resultCode != RESULT_OK) {
            return;
        }
        MediaProjection mediaProjection = systemService.getMediaProjection(resultCode, data);
        if (mediaProjection != null) {
            Intent intent = new Intent(this, MediaProjrctService.class);
            AppUtils.setActivity(this);
            AppUtils.setMediaProjection(mediaProjection);
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
                startForegroundService(intent);
            } else {
                startService(intent);
            }
            ttsUtils.startSpeech(AppUtils.getString(context,R.string.screenCatTextSussce));
        } else {
            ttsUtils.startSpeech(AppUtils.getString(context,R.string.screenCatInitError));
        }
    }
    private boolean getPermission() {
        int READ = ContextCompat.checkSelfPermission(getApplicationContext(), permissions[0]);
        int WRIT = ContextCompat.checkSelfPermission(getApplicationContext(), permissions[1]);
        if (READ != PackageManager.PERMISSION_GRANTED && WRIT != PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this, permissions, 200);
            return true;
        } else {
            return false;
        }
    }
    @Override
    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
        if (requestCode == 200 && grantResults[0] == PackageManager.PERMISSION_GRANTED && grantResults[1] == PackageManager.PERMISSION_GRANTED) {
            ttsUtils.startSpeech(AppUtils.getString(context,R.string.pressionOk));
        } else {
            ttsUtils.startSpeech(AppUtils.getString(context,R.string.pressionNO));
        }
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
    }
    @Override
    protected void onDestroy() {
        if (systemService != null) {
            systemService = null;
        }
        super.onDestroy();
    }
}

• m通信工程ainActivity首要有三个当地
• 1：获取截屏前台体系服务
• 2：翻开体系服务的辅助服务
• 3：获取保存图片的权限
• 4：接纳无障appstore碍服务发送的手势含义图解大全服务

总结

• 每个类之间的通讯
• 切换线程为了避免混淆 又改变为handler
• 在上一个版别后觉的手势操作比较契合操作。

软件架构

• 最好的规划便是没有规划，本项目是mvc架构。首要便是通信人家园服务层和数据层获取到相关内容告诉Activity进行更新。
• 没有过多的页面制作，首要的悉数在逻辑后台。纯离线方法的实时字幕辨认。

运用说明

• 1,翻开软件
• 2,点击获取两个权限
• 3,坚持后台运转
• 4,翻开相关视频app进入播映页面运通信大数据行程卡用手势开端读取字幕
• 未再横屏会提示先横屏在次软件工程操作就能够了

九,资源文件

• 首要是怕有些人不知道代码中http 404的意思 所以悉数放出来

<resources>
    <string name="app_name">字幕播报</string>
    <string name="accessblity">字幕播报</string>
    <string name="initAgin">初始化无障碍</string>
    <string name="initScreen">初始化读屏</string>
    <string name="action">开端运转</string>
    <string name="dupingpromising">初始化读屏所需求的权限，若不答应将会不行运用</string>
    <string name="wuzhangaipromising">初始无障碍所需求的权限，若不答应将会不行运用</string>
    <string name="save_img">保存辨认图片</string>
    <string name="guolv_danmu">过滤弹幕文字</string>
    <string name="window">实时悬浮窗</string>
    <string name="guolv_text">过滤满屏的弹幕信息，避免打乱辨认字幕准确。</string>
    <string name="save_text">保存辨认字幕过程中的图片。</string>
    <string name="helper">协助</string>
    <string name="pressionOk">权限获取成功</string>
    <string name="pressionNO">权限获取失利</string>
    <string name="screenReadingStart">现已开端读屏</string>
    <string name="screenReadingPaused">已进入竖屏,读屏现已暂停</string>
    <string name="screenReadingTip">当时竖屏状况,请进入视频播映页面横屏下运用读屏</string>
    <string name="screenReadingAction">已进入横屏,请运用手势开端读屏</string>
    <string name="screenCatDestroyError">服务异常退出，请查看服务权限后重新运用手势开端读屏。</string>
    <string name="notificationTitle">字幕辨认服务正在运转。</string>
    <string name="notificationText">可运用手势启用字幕播报。</string>
    <string name="notificationError">手势错误。</string>
    <string name="notificationLife">翻开保存辨认图片。</string>
    <string name="notificationNoLife">封闭保存辨认图片。</string>
    <string name="notificationRight">翻开过滤弹幕。</string>
    <string name="notificationNoRight">封闭过滤弹幕。</string>
    <string name="screenCatTextNoInit">字幕服务暂时没有初始化。</string>
    <string name="screenCatTextSussce">字幕服务初始化成功。</string>
    <string name="screenCatOnInit">字幕服务现已初始化。</string>
    <string name="screenCatInitError">字幕服务初始化失利。</string>
    <string name="usehelpr">再翻开无障碍功用后，在任何界面运用手势向下向左是翻开保存辨认图片，向下向右是过滤视频弹幕，向下向上是开端读取视频屏幕(仅限横屏运用)。</string>
</resources>

作业心得

说一下吧：我是一个应届毕业生，由于自己是专科，所以选择了一个薪软件库酬高的小公司，当初南京最楼房叫什么鹰的作业我没去，那时开的通信地址薪酬5000尽管低可是公司特别大，我现在懊悔没有去，去年导致我在小公司被裁员。本年我又手势密码踏入新手势变化的小公司，虽说薪酬高可是需求改变的是真快，我的网名便是由于这通信工程改的，还有一些无理的需求，代码是2015的通信技术耦合度高的离谱。这个项目便是公司要求做的手势舞视频 ，至今我无法检测到弹幕的位置，字幕我消耗了良久解决了，司理和我说：这个简单会很快，我想说关于我这样的没触摸过手势语NDK，也没触摸过AI的aappearancendapproveroid开发来说确实“很快” 两个星期写好，功能很差，后台一秒钟截一次图，然后交给NDK操作，这软件商店安装不消耗手机功能手势舞教学视频简单消耗什么，华为荣耀30我测试的时候卡关机很多次 ，而且短时刻内测试不出appointment来效果，测试便是一个很浪费时刻的问题，开端和我说辨认视频字幕，后httpwatch来又和我说辨认美国大片英文http协议字幕，好离谱。没有一个固定的需求，需求很乱，后来又说你慢。下次甘愿饿死街头也要找个需求稳定差不多的公司。
我不管谁看到，我也不管你是我领导仍是司理，我什么都不怕，能耐就开我。

提到司理，咱们公司没有产品司理，需求司理，总是靠嘴，假如靠嘴能够的话，我CV工程师能够靠嘴吗？

           ** 选对，比选的好重要，能够节约很多弯路，乃至时刻。**