The Research aimed to improve the inference efficiency of large language models; by optimizing the attention mechanism and KV Cache usage in the speculative decoding scenario, the project enhanced efficiency and reduced computational burden.

• Designed and implemented KV Cache optimization strategies; used pooling techniques and score-based attention selection mechanisms to effectively reduce cache occupancy.

• Managed the Token generation process, optimized attention distribution, improved the model’s inference accuracy and responsiveness.

• Conducted in-depth analysis of the Draft Model; identified and implemented optimization and deletion strategies for KV Cache to enhance model performance.

The research explores integrating generative AI into education, focusing on a proposed NSF project to implement this technology in classrooms. It examines behavioral research on human-technology interactions, aiming to enhance learning through AI. The goal is to optimize AI’s role in supporting and improving educational outcomes across different learning contexts.

• Integrated multiple cutting-edge technology stacks; automated project deployment with Vercel and GitHub Actions, which saved 32 working hours daily and effectively processed over 8,000 requests, significantly improving development efficiency and system stability.

• Developed identity authentication middleware, integrated it with the school’s SSO system, and used JWT for identity verification; built an RBAC-based access control system that comprehensively managed interface calls, user permissions, and access control, successfully preventing over 300 unauthorized access attempts.

• Embedded data into Pinecone namespaces, implemented a multi-tenant system, improved data retrieval efficiency, and reduced retrieval time from 45 seconds to 10 seconds.

• Developed the Retrieve Merger feature and optimized the effectiveness of Retriever in LangChain; achieved multi-source queries and provided accurate answers, improved the proxy response accuracy rate for complex multi-source queries by 25%, hence increasing user satisfaction by 20%.

• AI Middle Platform:

• Developed a text similarity comparison feature using the BERT model, which achieved a 95% accuracy rate, reducing processing time from 3 hours to 2 minutes.
• Applied LoRA and P-Tuning techniques in fine-tuning the ChatGLM-6B model on a private dataset, increasing QA accuracy by 15% and user satisfaction to 84.8%.
• Constructed a Kafka real-time data streaming system, optimized the cost of data processing and storage by 60%; the system processes over 20 GB of data daily.

• R&D Platform:

• Participated in the design of the micro-frontend architecture; built a component library with Vue.js, optimized the implementation with Nginx, Docker, and Jenkins, established coding standards, accelerated development cycle by 40%, reduced implementation time by 50%, served 500,000 users, and completed and delivered 43 projects annually.

• Marketing Management System:

• Served as the front-end team leader, led a team of 11 people to complete the development of over 100 pages and 8 major modules within three months; applied virtual scrolling and lazyload techniques, which reduced page loading time by 90%; improved product quality and user experience in a data-driven manner.

• Led threat assessment initiatives and formulated security policies targeting software vulnerabilities; conducted comprehensive analyses on vulnerability formation and impact, enhancing organizational defense capabilities.
• Applied advanced techniques such as fuzz testing, dynamic and static analysis, and symbolic execution using tools like AFL, Panda, IDA Pro, and S2E to identify critical features in the exploitation process and mitigate potential threats.
• Improved the efficiency of AFL fuzz testing and vulnerability detection processes by optimizing mutation strategies and program analysis, leading to a higher rate of target path activation and overall system security.

• Designed and developed a Use Case Analysis report and a detailed roadmap for IBM Watson’s digital twin technology. Created a prototype using React.js for the front end and GoLang with Gin for the backend, integrating RESTful APIs for seamless data communication.

• Conducted competitive analysis on potential applications, market viability, and technical aspects, using Pandas for data analysis and Tableau for data visualization. Provided strategic insights that led to the adoption of new features, generating $108,363 in total revenue.