Skip to main content

Show HN: How to use ChatGPT+ARKit to script experiences with natural language https://ift.tt/NmSvDVj

Show HN: How to use ChatGPT+ARKit to script experiences with natural language Video demonstration: https://youtu.be/bJFa5HOmhl4 I created a demo involving ChatGPT, OpenAI's Whisper running on-device for speech-to-text (because why not?), and ARKit. By defining a custom JavaScript environment with JavaScriptCore and describing it to ChatGPT, I was able to get it to produce functioning scripts that instantiate and manipulate 3D objects. Sketchfab is used to import 3D assets on the fly. Google Poly had a much better catalog of assets for this sort of thing but sadly, it's no longer available. The README.md describes how it all works but the gist of it is that user prompts are wrapped in a larger prompt that describes the environment (i.e., which functions are available to use and how they work) and constraints. The code is then executed directly. Assets for objects are fetched as needed by searching Sketchfab. https://ift.tt/CRydSLn December 21, 2022 at 11:51PM
Labels:

Comments

Post a Comment

Popular posts from this blog

Show HN: TypeScript query builder with full type inference https://ift.tt/xZp9HOm

Show HN: TypeScript query builder with full type inference Hey HN! Colin here - a TypeScripter, open sourcer, and engineer at EdgeDB. As the creator of Zod and tRPC, I'm interested in designing tools/APIs that use type inference and generics to make life easier for devs. This query builder represents another step in that direction. We set out to build an EdgeQL query builder that can express queries of arbitrary complexity (EdgeQL has feature parity with SQL, roughly) and infer the static type of the query result. We introspect the database and generate a schema-aware client that represent any query, including ones that use built-in functions, operators, string/array/tuple indexing, aggregations, conditionals, type casting, subqueries, computed properties, etc—things most ORMs can’t represent. This post mostly discusses the API design, which I think will be interesting regardless of familiarity with EdgeQL. I’d love to see some of these ideas bleed into future generations of TypeSc...
Post a Comment
Read more