Locker.ai is a team project that connects lost-property registration, search, identity verification, and return through an LLM and smart lockers. As a member of the National Institute of Technology (KOSEN), Ibaraki College team whose official Japanese name is “明日のDCON楽しみだね,” I was primarily responsible for the web frontend, conversational AI interface, and API integration. Hardware, machine learning, business planning, and the presentation were shared across the team, and our DCON results were achieved as a team.
We continued development after presenting the research and a demonstration at IPSJ Interaction 2024 in March 2024, eventually reaching the final of the National Institute of Technology Deep Learning Contest 2025 (DCON2025) in May 2025. This page brings the presentation, implementation, and judging experience together as one record.
Product Presentation Overview
We began with the time and manual work required from everyone involved: the person who finds an item, the facility that stores it, and the person searching for it. Locker.ai was designed so that a finder could register an item on the web and leave it in a nearby locker, while the owner narrowed down candidates through chat, completed identity verification, and collected the item from the specified locker.
【茨城工業高等専門学校】Locker.ai：LLM×スマートロッカーによる自動応対遺失物管理サービス学校名茨城工業高等専門学校 チーム名明日のDCON楽しみだね 作品名Locker.ai：LLM×スマートロッカーによる自動応対遺失物管理サービス メンター折茂 美保氏（ボストン コンサルティング グループ マネージdcon.ai
National Hitachi Seaside Park, JR East Katsuta Station, and the Ibaraki Prefectural Police also supported our problem research through questionnaires. We treated reporting by the finder, storage by the facility, and enquiries from the owner as separate parts of the process, then examined which manual steps could be reduced by combining the web application with lockers.
The 39-slide finals deck begins with the problems faced by finders, facility operators, and owners in the conventional process. It then introduces item registration and conversational enquiry, the LLM, the team's own model, multilingual support, and modular smart lockers. The final section presents the B2B2C and B2G2C business models and an adoption roadmap. Rather than presenting the technology in isolation, the deck follows one story: whose work is replaced, where that change occurs, and how the service could scale to facilities of different sizes.
The deck states the goal of allowing finders and owners to complete the lost-property process themselves. That does not mean that facility operators have already become unnecessary. Collecting stored items, handling exceptions, and stopping an incorrect return still require an operational process. The presentation communicates the value of automation, while the implementation keeps a search match separate from the state in which an item is ready for collection.
The business model assumes that a facility or local government is the contracting customer, while finders and owners use the web application and lockers in a B2B2C or B2G2C relationship. The modular locker design lets the number of compartments and the scope of deployment change with the size and existing operating model of each facility.
Technology
The Team's Overall System
The public repository and presentation materials document the following stack at the time of the DCON2025 finals:
- Web: Next.js 14, React 18, TypeScript, next-intl, Vercel AI SDK 4, and GPT-4.1
- API: NestJS 10, Apollo GraphQL, Prisma, and GraphQL Code Generator
- Data and authentication: Supabase Auth, Storage, Postgres, and pgvector
- Similarity inference: a lost-item matching service built with FastAPI and PyTorch
- Hardware: a Raspberry Pi 5 control server, Raspberry Pi Pico lock modules, solenoid locks, and a QR-code reader
- Development: pnpm workspaces, Turborepo, Jest, Storybook, and Chromatic
- Deployment: Vercel for the web application and Railway for the API and inference service
This is the team's overall technology stack. I was not solely responsible for every selection or implementation.
The public monorepo separates the user-facing website, the locker-dashboard displayed at the locker, the NestJS api, and a UI catalogue, managed through pnpm workspaces and Turborepo. Based on the last commit before the finals on 9 May 2025, the main processing boundaries were as follows.
Turning Reported Photos into Searchable Data
When a finder uploads photos, the API stores them in Supabase Storage and passes the same images to GPT-4.1. The model generates an English feature description covering visible attributes such as colour, shape, material, brand, model number, and surface text. A Japanese translation is generated for display, while the English description is embedded with text-embedding-3-small and stored in the Postgres vector column for retrieval.
The owner's description is also translated into English and passed through the same embedding model. The public implementation calculates similarity from pgvector's cosine-distance operator, <=>, and retrieves candidates among items that do not yet have an owner.
SELECT
id,
1 - (embedded_description <=> ${embeddedDescription}::vector) AS similarity
FROM public.lost_items
WHERE owner_id IS NULL
LIMIT 10Text similarity is not the final decision. The NestJS API sends each candidate's similarity and the difference between the reported loss date and item report date to the identification-neural-network API, which returns match and rejection probabilities. The FastAPI and PyTorch training implementation is not included in the public monorepo, but its client contract confirms the similarity and date_difference inputs and the pair of probability outputs.
Keeping Search Separate from LLM Output
In the search chat that I worked on, generated text and database search are separate operations. A Next.js Route Handler streams GPT-4.1 responses with the Vercel AI SDK's streamText. A similar-item query is sent through GraphQL only when the model calls searchLostItem.
const result = streamText({
model: openai("gpt-4.1"),
system: `${SYSTEM_PROMPT}\n\n${languageInstructions[locale]}`,
messages,
tools,
})
return result.toDataStreamResponse()The description passed to the search tool is translated into English to match the internally stored data, while the response follows the user's language. The assistant asks for more characteristics when a description is ambiguous and converts relative dates into absolute dates before matching. Separating the visible conversation, tool call, and GraphQL query lets the model use retrieved results to formulate its next question rather than inventing a candidate.
Each candidate includes approval and rejection probabilities. In the finals implementation, a rejection probability of 0.4 or higher prevented the interface from disclosing the candidate image or details and instead prompted the user for more information. Preventing an incorrect return requires more than this threshold: owner-only characteristics, authentication, and conditions for operator intervention all need to work together.
Connecting the Web Application to a Physical Locker
Finding a candidate and being authorised to open a locker are separate states. I implemented a QR-code authentication dialog, a header that reflects authentication state, a screen showing the relevant locker location, and the flow from a search result to a collection request.
The web application communicates with the NestJS API through GraphQL, with generated types kept in sync by GraphQL Code Generator. Before the finals, I also updated internationalisation across the web application and API, related GraphQL mutations, and database migrations. User-facing text, API types, stored data, and physical-locker state therefore had to be treated as one monorepo change rather than an isolated screen update.
Each item stores reportedAt, deliveredAt, ownedAt, and retrievedAt, while the user's lostAndFoundState records which action they are currently completing. Creating a report moves a finder to DELIVERING; placing the item in an empty drawer returns that state to NONE. Claiming an item moves its owner to RETRIEVING; removing it from the drawer completes the process.
On the locker side, the device posts its status and a fingerprint challenge to NestJS, which publishes updates to the dashboard through a GraphQL Subscription. When an unlinked fingerprint is read, the dashboard shows a QR code containing the lockerId and hashed fingerprint ID. An authenticated user scans it to associate the fingerprint with their Supabase account and clear the outstanding challenge.
The IPSJ Interaction 2024 paper reports that VLM-generated feature descriptions were often close to human descriptions, while also documenting cases in which they performed worse. Adding a model does not make the return process safe by itself. Candidate presentation and identity verification must remain separate, and the system must be able to stop a return when a component fails.
interaction-ipsj.orginteraction-ipsj.orgAt DCON2025
DCON evaluates not only the technology in projects that combine engineering and deep learning, but also their effect on a social problem and their business potential, expressed through a corporate valuation. During the technical review on the day before the final presentation, we operated the hardware and system while explaining the AI architecture, smart-locker integration, and safety considerations for real-world use.
The behind-the-scenes video from Minna no Kosen Channel shows our technical review from 12:47. It records not only the finished demonstration but also the distance between the team and the reviewers as we explain the implementation and move through the venue.
Mentorship and the Business Proposal
DCON assigns a mentor to each of the ten finalist teams. Locker.ai's mentor was Miho Orimo, Managing Director & Partner at Boston Consulting Group and the Japan leader of its Social Impact practice. Her areas of expertise include the public sector, education, climate and sustainability. Her BCG profile also records experience supporting strategy execution for central government, universities, and municipalities, as well as new-business development for companies.
Ms Orimo accompanied the team as we prepared for a final that assessed not only engineering quality, but also who would pay for the service, where deployment should begin, and how the business could expand. The team representative's published comment thanks her alongside the facilities and public organisations that supported the questionnaires.
The finals segment proceeds from a mentor introduction to a five-minute team presentation, technical feedback, questions from the judges, and investment indications. The official archive starts the Locker.ai segment at 3:32:07. It preserves the live proposal and questions around a product designed to connect to working hardware—context that the slides alone cannot show.
Forty-two National Institutes of Technology and 95 teams entered DCON2025. Locker.ai placed fifth in the final judging with a corporate valuation of JPY 50 million. The project also received three company-sponsored awards: the Topy Industries Award (トピー工業賞), the Hitachi Industry & Control Solutions Award (日立産業制御ソリューションズ賞), and the QUICK Award (QUICK賞). These results evaluated the team's integrated work across hardware, AI, validation, business proposal, and presentation—not the web implementation alone.
Developing the system through the finals taught me that model accuracy alone cannot explain product safety. Can an incorrect candidate be stopped before return? Is candidate information disclosed only when necessary? Does the system return to a safe state when the AI or network fails? These remain acceptance criteria that must be considered across the web application, API, AI services, and locker hardware.
Sources
- Locker.ai public repository
- Commits by Reo Hakuta
- IPSJ Interaction 2024 paper (PDF)
- DCON2025 Locker.ai team page
- DCON2025 final results
- DCON2025 event report
- Finals result published by National Institute of Technology, Ibaraki College
- Miho Orimo profile at Boston Consulting Group
- DCON2025 Finals Announcement