1.前言

近两年来工信部关于运用的隐私合规安全问题更加重视，对 Android 平台的管控程度也要比 IOS 平台严厉许多，许多不合规的运用也先后被下架要求整改。笔者就曾遇到过加班整改隐私合规的问题，隐私合规问题主要针对两个方面。

• 在用户赞同隐私协议之前不能搜集用户隐私数据，例如 IMEI、AndroidId、MAC 等
• 在用户赞同隐私协议之后，搜集用户数据行为在对应场景不能超频。比如一分钟不能超过 3 次获取 IMEI

针对上述两个方面，有以下办法来针对

• 经过静态扫描，搜集项目中（自有代码 + 三方 sdk）运用隐私合规相关 api 的相关代码
• 经过 ASM 插桩，在调用隐私合规 api 之前刺进代码，记载运行时的办法调用链和当时时刻
• hook 隐私合规 api，替换字节码指令将调用链指向东西类，在未赞同隐私协议之前，不调用相关的 api

2.完结

2.1 注解和东西类

经过界说注解和东西类，用来界说要处理哪些隐私合规相关的办法，现在笔者现已处理了大部分，请放心食用

2.1.1 注解

/**
 * 搜集和注解匹配的办法
 * visitMethodInsn(int opcode, String owner, String name,String desc)
 *
 * ======假如 originName 和 originDesc 传""，逻辑会在插件中处理=====
 */
@Retention(RetentionPolicy.CLASS)
@Target({ElementType.METHOD})
public @interface AsmMethodReplace {
    /**
     * 指令操作码
     */
    int targetMethodOpcode();
    /**
     * 办法一切者类
     */
    String targetClass();
    /**
     * 办法名称
     */
    String targetName() default "";
    /**
     * 办法描述符
     */
    String targetDesc() default "";
    /**
     * 是否进行 hook
     */
    boolean hook() default false;
}

该注解用来匹配调用隐私合规 api 的字节码指令，例如经过 ASM 调用 getImei 的字节码指令为 methodVisitor.visitMethodInsn(INVOKEVIRTUAL, "android/telephony/TelephonyManager", "getImei", "()Ljava/lang/String;", false);。targetName 特点和 targetDesc 能够不赋值，这儿做了取巧的处理，会依据东西类的 method name 和method descriptor 推断出调用隐私合规办法的字节码指令，这块后边插件会处理。 最后的 hook 特点表明是否 hook 掉原始的调用链，将调用指向东西类中的办法。

2.1.2 东西类

上述注解能够用在任何地方，笔者将常用的隐私合规的办法聚合起来，办法东西类中统一处理，例如处理 IMEI 的逻辑如下：

@RequiresApi(api = Build.VERSION_CODES.O)
@AsmMethodReplace(targetMethodOpcode = OPCODE_INVOKEVIRTUAL
        , targetClass = CLASS_NAME_TELEPHONYMANAGER,hook = true)
public static String getImei(TelephonyManager telephonyManager) {
    if (!checkAgreePrivacy("getImei")) {
        Log.e(TAG, TIP);
        return "";
    }
    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
        Log.i(TAG, "getImei-SDK_INT above android Q");
        return "";
    }
    return telephonyManager.getImei();
}

假如还没有赞同隐私协议，直接 return “”，否者走正常的调用办法。一起，经过东西类

@Override
   protected void attachBaseContext(Context base) {
       super.attachBaseContext(base);
       ConfigGlobal.getInstance().setStoreDirectory(base.getExternalCacheDir().getAbsolutePath());
   }

设置存储调用仓库和时刻的文件途径。

2.2 插件处理

gradle 插件的基本运用在这儿就不赘述了，这儿主要是两方面的处理

• 编译时扫描代码，处理有界说特定注解的办法，剖析字节码指令，搜集一切需求处理的隐私合规相关 api 相关的信息
• 再次扫描，依据第一次扫描搜集到的信息，判别当时类是否含有调用隐私合规 api 的字节码指令，假如有，在该类中注入一个写文件办法及在隐私合规 api 调用指令之前刺进写文件的字节码指令，用来记载调用仓库和频次。

2.2.1 模版代码

下面这块代码是我们在自界说 gradle 插件时常用的模版代码，供我们运用

package com.zhangyue.ireader
import com.android.build.api.transform.*
import com.android.build.gradle.internal.pipeline.TransformManager
import com.zhangyue.ireader.plugin_privacy.PrivacyGlobalConfig
import com.zhangyue.ireader.util.CommonUtil
import com.zhangyue.ireader.util.Logger
import org.apache.commons.io.FileUtils
import org.apache.commons.io.IOUtils
import org.gradle.api.Project
import java.util.concurrent.AbstractExecutorService
import java.util.concurrent.Callable
import java.util.concurrent.ForkJoinPool
import java.util.jar.JarEntry
import java.util.jar.JarFile
import java.util.jar.JarOutputStream
abstract class BaseTransform extends Transform {
    AbstractExecutorService executorService = ForkJoinPool.commonPool()
    private List<Callable<Void>> taskList = new ArrayList<>()
    protected Project project
    BaseTransform(Project project) {
        this.project = project
    }
    @Override
    String getName() {
        return getClass().simpleName
    }
    @Override
    Set<QualifiedContent.ContentType> getInputTypes() {
        return TransformManager.CONTENT_CLASS
    }
    @Override
    Set<? super QualifiedContent.Scope> getScopes() {
        return TransformManager.SCOPE_FULL_PROJECT
    }
    @Override
    boolean isIncremental() {
        return true
    }
    @Override
    void transform(TransformInvocation transformInvocation) throws TransformException, InterruptedException, IOException {
        super.transform(transformInvocation)
        println("transform start--------------->")
        if (firstTransform()) {
            printCopyRight()
        }
        onTransformStart(transformInvocation)
        def startTime = System.currentTimeMillis()
        def inputs = transformInvocation.inputs
        def outputProvider = transformInvocation.outputProvider
        def context = transformInvocation.context
        def isIncremental = transformInvocation.isIncremental()
        if (!isIncremental) {
            outputProvider.deleteAll()
        }
        //1
//        inputs.each { input ->
//            input.jarInputs.each { JarInput jarInput ->
//                forEachJar(jarInput, outputProvider, context, isIncremental)
//            }
//
//            input.directoryInputs.each { DirectoryInput dirInput ->
//                forEachDir(dirInput, outputProvider, context, isIncremental)
//            }
//        }
        //3
        inputs.each { input ->
            input.jarInputs.each { jarInput ->
                submitTask(new Runnable() {
                    @Override
                    void run() {
                        forEachJar(jarInput, outputProvider, context, isIncremental)
                    }
                })
            }
            input.directoryInputs.each { DirectoryInput dirInput ->
                submitTask(new Runnable() {
                    @Override
                    void run() {
                        forEachDir(dirInput, outputProvider, context, isIncremental)
                    }
                })
            }
        }
        def futures = executorService.invokeAll(taskList)
        futures.each { it ->
            it.get()
        }
        onTransformEnd(transformInvocation)
        println(getName() + "transform end--------------->" + "duration : " + (System.currentTimeMillis() - startTime) + " ms")
    }
    void submitTask(Runnable runnable) {
        taskList.add(new Callable<Void>() {
            @Override
            Void call() throws Exception {
                runnable.run()
                return null
            }
        })
    }
    void forEachDir(DirectoryInput directoryInput, TransformOutputProvider outputProvider, Context context, boolean isIncremental) {
        def inputDir = directoryInput.file
        File dest = outputProvider.getContentLocation(
                directoryInput.name,
                directoryInput.contentTypes,
                directoryInput.scopes,
                Format.DIRECTORY
        )
        println "directoryInputPath:" + directoryInput.file.absolutePath
        println "destPath:" + dest.absolutePath
        def srcDirPath = inputDir.absolutePath
        def destDirPath = dest.absolutePath
        def temporaryDir = context.temporaryDir
        FileUtils.forceMkdir(dest)
        Logger.info("srcDirPath:${srcDirPath}, destDirPath:${destDirPath}")
        if (isIncremental) {
            directoryInput.getChangedFiles().each { entry ->
                def classFile = entry.key
                switch (entry.value) {
                    case Status.NOTCHANGED:
                        Logger.info("处理 class： " + classFile.absoluteFile + " NOTCHANGED")
                        break
                    case Status.REMOVED:
                        Logger.info("处理 class： " + classFile.absoluteFile + " REMOVED")
                        //终究文件应该寄存的途径
                        def destFilePath = classFile.absolutePath.replace(srcDirPath, destDirPath)
                        def destFile = File(destFilePath)
                        if (destFile.exists()) {
                            destFile.delete()
                        }
                        break
                    case Status.ADDED:
                    case Status.CHANGED:
                        Logger.info("处理 class： " + classFile.absoluteFile + " ADDED or CHANGED")
                        modifyClassFile(classFile, srcDirPath, destDirPath, temporaryDir)
                        break
                    default:
                        break
                }
            }
        } else {
            com.android.utils.FileUtils.getAllFiles(inputDir).each { File file ->
                modifyClassFile(file, srcDirPath, destDirPath, temporaryDir)
            }
        }
    }
    void modifyClassFile(classFile, srcDirPath, destDirPath, temporaryDir) {
        Logger.info("处理 class： " + classFile.absoluteFile)
        //目标途径
        def destFilePath = classFile.absolutePath.replace(srcDirPath, destDirPath)
        def destFile = new File(destFilePath)
        if (destFile.exists()) {
            destFile.delete()
        }
        Logger.info("处理 class： destFile" + destFile.absoluteFile)
        String className = CommonUtil.path2ClassName(classFile.absolutePath.replace(srcDirPath + File.separator, ""))
        Logger.info("处理 className： " + className)
        File modifyFile = null
        if (CommonUtil.isLegalClass(classFile) && shouldHookClass(className)) {
            modifyFile = getModifyFile(classFile, temporaryDir, className)
        }
        if (modifyFile == null) {
            modifyFile = classFile
        }
        FileUtils.copyFile(modifyFile, destFile)
    }
    File getModifyFile(File classFile, File temporaryDir, String className) {
        byte[] sourceBytes = IOUtils.toByteArray(new FileInputStream(classFile))
        def tempFile = new File(temporaryDir, CommonUtil.generateClassFileName(classFile))
        if (tempFile.exists()) {
            FileUtils.forceDelete(tempFile)
        }
        def modifyBytes = modifyClass(className, sourceBytes)
        if (modifyBytes == null) {
            modifyBytes = sourceBytes
        }
        tempFile.createNewFile()
        def fos = new FileOutputStream(tempFile)
        fos.write(modifyBytes)
        fos.flush()
        IOUtils.closeQuietly(fos)
        return tempFile
    }
    void forEachJar(JarInput jarInput, TransformOutputProvider outputProvider, Context context, boolean isIncremental) {
        Logger.info("jarInput:" + jarInput.file)
        File destFile = outputProvider.getContentLocation(
                //避免同名被覆盖
                CommonUtil.generateJarFileName(jarInput.file), jarInput.contentTypes, jarInput.scopes, Format.JAR)
        //增量编译处理
        if (isIncremental) {
            Status status = jarInput.status
            switch (status) {
                case Status.NOTCHANGED:
                    Logger.info("处理 jar： " + jarInput.file.absoluteFile + " NotChanged")
                    //Do nothing
                    return
                case Status.REMOVED:
                    Logger.info("处理 jar： " + jarInput.file.absoluteFile + " REMOVED")
                    if (destFile.exists()) {
                        FileUtils.forceDelete(destFile)
                    }
                    return
                case Status.ADDED:
                case Status.CHANGED:
                    Logger.info("处理 jar： " + jarInput.file.absoluteFile + " ADDED or CHANGED")
                    break
            }
        }
        if (destFile.exists()) {
            FileUtils.forceDelete(destFile)
        }
        CommonUtil.isLegalJar(jarInput.file) ? transformJar(jarInput.file, context.getTemporaryDir(), destFile)
                : FileUtils.copyFile(jarInput.file, destFile)
    }
    def transformJar(File jarFile, File temporaryDir, File destFile) {
        Logger.info("处理 jar： " + jarFile.absoluteFile)
        File tempOutputJarFile = new File(temporaryDir, CommonUtil.generateJarFileName(jarFile))
        if (tempOutputJarFile.exists()) {
            FileUtils.forceDelete(tempOutputJarFile)
        }
        JarOutputStream jarOutputStream = new JarOutputStream(new FileOutputStream(tempOutputJarFile))
        JarFile inputJarFile = new JarFile(jarFile, false)
        try {
            def entries = inputJarFile.entries()
            while (entries.hasMoreElements()) {
                def jarEntry = entries.nextElement()
                def entryName = jarEntry.getName()
                def inputStream = inputJarFile.getInputStream(jarEntry)
                try {
                    byte[] sourceByteArray = IOUtils.toByteArray(inputStream)
                    def modifiedByteArray = null
                    if (!jarEntry.isDirectory() && CommonUtil.isLegalClass(entryName)) {
                        String className = CommonUtil.path2ClassName(entryName)
                        if (shouldHookClass(className)) {
                            modifiedByteArray = modifyClass(className, sourceByteArray)
                        }
                    }
                    if (modifiedByteArray == null) {
                        modifiedByteArray = sourceByteArray
                    }
                    jarOutputStream.putNextEntry(new JarEntry(entryName))
                    jarOutputStream.write(modifiedByteArray)
                    jarOutputStream.closeEntry()
                } finally {
                    IOUtils.closeQuietly(inputStream)
                }
            }
        } finally {
            jarOutputStream.flush()
            IOUtils.closeQuietly(jarOutputStream)
            IOUtils.closeQuietly(inputJarFile)
        }
        FileUtils.copyFile(tempOutputJarFile, destFile)
    }
    private byte[] modifyClass(String className, byte[] sourceBytes) {
        byte[] classBytesCode
        try {
            classBytesCode = hookClassInner(className, sourceBytes)
        } catch (Throwable e) {
            e.printStackTrace()
            classBytesCode = null
            println "throw exception when modify class ${className}"
        }
        return classBytesCode
    }
    /**
     * 打印日志信息
     */
    static void printCopyRight() {
        println()
        println '#######################################################################'
        println '##########                                                    '
        println '##########                欢迎运用隐私合规处理插件'
        println '##########                                                    '
        println '#######################################################################'
        println '##########                                                    '
        println '##########                 插件装备参数                         '
        println '##########                                                    '
        println '##########                -isDebug: ' + PrivacyGlobalConfig.isDebug
        println '##########                -handleAnnotationName: ' + PrivacyGlobalConfig.handleAnnotationName
        println '##########                -exclude: ' + PrivacyGlobalConfig.exclude
        println '##########                                                    '
        println '##########                                                    '
        println '##########                                                    '
        println '#######################################################################'
        println()
    }
    protected boolean firstTransform() {
        return false
    }
    boolean shouldHookClass(String className) {
        def excludes = PrivacyGlobalConfig.exclude
        if (excludes != null) {
            for (String string : excludes) {
                if (className.startsWith(string)) {
                    return false
                }
            }
        }
        return shouldHookClassInner(className)
    }
    protected abstract boolean shouldHookClassInner(String className)
    protected abstract byte[] hookClassInner(String className, byte[] bytes)
    protected abstract void onTransformStart(TransformInvocation transformInvocation)
    protected abstract void onTransformEnd(TransformInvocation transformInvocation)
}

2.2.2 注解处理 transform

搜集和处理具有特定注解的字节码指令,给下一个 transform 运用

@Override
byte[] hookClassInner(String className, byte[] bytes) {
    ClassReader cr = new ClassReader(bytes)
    ClassNode classNode = new ClassNode()
    cr.accept(classNode, 0)
    classNode.methods.each { methodNode ->
        //编译期注解
        methodNode.invisibleAnnotations.
                each { annotationNode ->
                    if (PrivacyGlobalConfig.getHandleAnnotationName() == annotationNode.desc) {
                        collectPrivacyMethod(annotationNode, methodNode, cr.className)
                    }
                }
    }
    return bytes
}
/**
 * 搜集注解和注解相关的办法
 * @param annotationNode 注解信息
 * @param methodNode 办法信息
 */
static collectPrivacyMethod(AnnotationNode annotationNode, MethodNode methodNode, String className) {
    List<Object> values = annotationNode.values
    Logger.info("annotation values : ${values}")
    MethodReplaceItem item = new MethodReplaceItem(values, methodNode, CommonUtil.getClassInternalName(className))
    PrivacyGlobalConfig.methodReplaceItemList.offer(item)
    Logger.info("collectPrivacyMethod success: ${item}")
    println("collectPrivacyMethod success: ${item}")
}

MethodReplaceItem中封装了搜集到的字节码特点，一起会依据注解相关办法的字节码指令推断出想要处理的隐私合规 api 的字节码指令。

MethodReplaceItem(List<Object> annotationPair, MethodNode methodNode, String owner) {
    replaceOpcode = Opcodes.INVOKESTATIC
    replaceClass = owner
    replaceMethod = methodNode.name
    replaceDesc = methodNode.desc
    for (int i = 0; i < annotationPair.size(); i = i + 2) {
        def key = annotationPair[i]
        def value = annotationPair[i + 1]
        if (key == "targetMethodOpcode") {
            targetOpcode = value
        } else if (key == "targetClass") {
            targetOwner = value
        } else if (key == "targetName") {
            targetMethod = value
        } else if (key == "targetDesc") {
            targetDesc = value
        }else if(key == "hook"){
            willHook = value
        }
    }
    Logger.info("=====targetOpcode:${targetOpcode},targetOwner:${targetOwner} , replaceDesc${replaceDesc}")
    if (isEmpty(targetMethod)) {
        targetMethod = replaceMethod
    }
    if (isEmpty(targetDesc)) {
        //静态办法，oriDesc 跟 targetDesc 一样
        if (targetOpcode == Opcodes.INVOKESTATIC) {
            targetDesc = replaceDesc
        } else {
            //非静态办法，约定第一个参数是实例类名，oriDesc 比 targetDesc 少一个参数，处理一下
            // (Landroid/telephony/TelephonyManager;)Ljava/lang/String ->  ()Ljava/lang/String
            def param = replaceDesc.split('\)')[0] + ")"
            def result = replaceDesc.split('\)')[1]
            def index = replaceDesc.indexOf(targetOwner)
            if (index != -1) {
                param = "(" + param.substring(index + targetOwner.length() + 1)
            }
            Logger.info("index::: ${index}")
            targetDesc = param + result
        }
    }
}

2.2.3 合规办法处理 transform

再次扫描整个项目，依据在上一个 transform 中搜集到的要处理的隐私合规的 api，遍历字节码指令，当匹配上时，在当时的类中注入写文件的办法，一起在调用隐私合规的字节码指令前刺进写文件的字节码指令，用来记载。

@Override
byte[] hookClassInner(String className, byte[] bytes) {
    Logger.info("${getName()} modifyClassInner--------------->")
    def findHookPoint = false
    Map<MethodNode, InsertInsnPoint> collectMap = new HashMap<>()
    ClassReader cr = new ClassReader(bytes)
    ClassNode classNode = new ClassNode()
    cr.accept(classNode, ClassReader.EXPAND_FRAMES)
    classNode.methods.each { methodNode ->
        //过滤掉含有特定注解的办法
        if (isNotHookMethod(cr.className, methodNode)) {
            methodNode.instructions.each { insnNode ->
                //判别字节码能否匹配
                def methodReplaceItem = searchHookPoint(insnNode)
                if (methodReplaceItem != null) {
                    //判别是否需求 hook 掉当时指令
                    def inject = methodReplaceItem.willHook
                    //记载隐私合规 api 所在的类及办法
                    logHookPoint(classNode.name, methodReplaceItem, methodNode, insnNode.opcode, insnNode.owner, insnNode.name, insnNode.desc, inject)
                    if (inject) {
                        //hook
                        injectInsn(insnNode, methodReplaceItem)
                    }
                    //刺进写文件办法指令，搜集调用隐私办法的仓库
                    collectInsertInsn(insnNode, methodNode, classNode, collectMap, inject)
                    findHookPoint = true
                }
            }
        }
    }
    if (!collectMap.isEmpty() && findHookPoint) {
        //刺进写文件指令，用来展示仓库信息
        collectMap.each { key, value ->
            key.instructions.insert(value.hookInsnNode, value.instList)
        }
        //刺进 writeToFile 办法
        ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_MAXS)
        classNode.accept(cw)
        insertWriteToFileMethod(cw)
        return cw.toByteArray()
    }
    return bytes
}

在 collectInsertInsn 办法中，经过 throwable 来搜集当时的仓库

/**
 * 搜集 在调用特定的办法前刺进调用写入文件的办法的指令
 * @param insnNode
 * @param methodNode
 * @param classNode
 * @param collectMap
 */
static void collectInsertInsn(insnNode, methodNode, classNode, collectMap, Inject) {
    def className = classNode.name
    def methodName = methodNode.name
    def methodDesc = methodNode.desc
    def owner = null
    def name = null
    def desc = null
    if (insnNode instanceof MethodInsnNode) {
        owner = insnNode.owner
        name = insnNode.name
        desc = insnNode.desc
    }
    //------log
    StringBuilder lintLog = new StringBuilder()
    lintLog.append(className)
    lintLog.append("  ->  ")
    lintLog.append(methodName)
    lintLog.append("  ->  ")
    lintLog.append(methodDesc)
    lintLog.append("\r\n")
    lintLog.append(owner)
    lintLog.append("  ->  ")
    lintLog.append(name)
    lintLog.append("  ->  ")
    lintLog.append(desc)
    //------要刺进字节码指令
    lintLog.append("\r\n")
    InsnList insnList = new InsnList()
    insnList.add(new LdcInsnNode(lintLog.toString()))
    insnList.add(new TypeInsnNode(Opcodes.NEW, "java/lang/Throwable"))
    insnList.add(new InsnNode(Opcodes.DUP))
    insnList.add(new MethodInsnNode(Opcodes.INVOKESPECIAL, "java/lang/Throwable", "<init>", "()V", false))
    insnList.add(new MethodInsnNode(Opcodes.INVOKESTATIC, className, writeToFileMethodName, writeToFileMethodDesc))
    println "刺进指令完结 =---------->"
    collectMap.put(methodNode, new InsertInsnPoint(insnList, insnNode))
}

终究，在项目编译完结之后，会在项目的根目录下生成 replaceInsn.txt 文件，记载包含隐私合规 api 的类和相关办法。

当项目运行起来之后，会在设置的途径中（笔者设置在 getExternalCacheDir 中）生成 privacy_log.txt 文件，里面会记载隐私合规 api 的调用仓库和时刻，依据该调用链，我们就能够快速定位是哪一块事务执行了灵敏操作。

总结

经过 ASM + gradle plugin ，能够排查出大部分的隐私合规问题。有什么不足之处，也请读者多多提意见和主张。

源码

改项目现已在 github 上开源，期望我们多多围观。 github.com/season-max/…

参考链接： ASM hook隐私办法调用，避免App被下架

去哪儿 Android 客户端隐私安全处理计划

ASM 字节码插桩：助力隐私合规