学习实践

本地高端笔记本部署LLM:LLaMA3 7B

我非常重视理论与实践,常常关注国外人工智能论坛的研究报告,并了解到吴恩达教授对于本地运行大语言模型和自助代理的光明前景。因此,我决定在本地构建一个简单的人工智能助手。我已经在我的高性能笔记本电脑上成功部署了LLaMA3 7B开源模型,并进一步构建了个人知识库。通过使用RAG(Retrieval-Augmented Generation)方法,我打造了一个高效、专业且定制化的人工智能助手,用于高效检索和巩固我的个人知识图谱。与我之前通过Anaconda构建的独立环境不同,这次我使用了Docker构建了一个专用容器,用于下载并运行LLaMA3 7B模型。我通过Python脚本设置了整体运行流程,减轻了模型产生幻觉的问题。此外,我还集成了自动调用外部网站API的功能,能够实时获取信息,并读取各种本地文件,实现了更加高效的信息处理能力。

ChatGLM3-6B本地测试

LLaMA3 7B模型本地测试

Author

wwq136@163.com
I am a junior majoring in Measurement and Control Technology and Instrumentation, with a passion for exploration and innovation. I have participated in various competitions, such as the Electronic Design Competition and Intelligent Vehicle Competition, and have contributed to national innovation projects, including the development of a fruit-picking robot. I possess strong embedded systems skills, working with platforms like STM32, Arduino, Raspberry Pi, and Jetson Nano, alongside proficiency in C/C++ and Python programming. My interests lie in the intersection of robotics and artificial intelligence, particularly in areas like computer vision (CV) and natural language processing (NLP), with a focus on machine learning. I am constantly learning AI-related technologies and exploring ways to integrate them into practical applications, aspiring to develop digital twins incorporating AI. I use Ubuntu 22.04 both on WSL2 for Windows and on my server. Currently, I am preparing for the IELTS, aiming for a 7.0 score, with a current score of 6.5. My future academic goal is to study at a top 50 QS university in Australia, such as UNSW Sydney, the country’s top-ranked university for engineering. Post-graduation, I plan to pursue entrepreneurship, focusing on local AI model deployment, particularly in healthcare. In addition to my technical pursuits, I am fascinated by Western culture, history, and philosophy, and I look forward to returning to Europe to further explore these areas.
Total post: 52

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注