Palico App

This is the core framework for Palico App. This framework has the following responsibilities:

• Help developers define their LLM application
• Help developers define their experiments
• Setup an API for other services to communicate with the LLM application
• Manage database tables and schemas

Development

Make sure you have your workspace setup. If you haven't already, follow the Workspace Setup guide.

One-time Setup

To properly test our changes, we'll need to publish the palico-app package to a local npm registry, then consume it from the sample-app project. We'll use Verdaccio for this purpose.

1. Setup Verdaccio

   npm install -g verdaccio
   Copy

2. Point sample-app to Verdaccio:

   cd sample-app
   echo "registry=https://registry.npmjs.org/" >> ".npmrc"
   echo "@palico-ai:registry=http://localhost:4873/" >> ".npmrc"
   Copy

3. Add a reinstall script in sample-app/package.json:

   "scripts": {
     "rebuild": "(cd ../palico-ai && nx run-many -t publish-local -p palico-app -p common)",
     "reinstall": "npm run rebuild && npm uninstall @palico-ai/app && npm install @palico-ai/app"
   }
   Copy

Testing Changes

1. Make sure Verdaccio is running in a separate console window

   verdaccio
   Copy

2. After you have made a change to packages/palico-app, publish these changes to sample-app:

   cd sample-app
   npm run reinstall
   Copy

3. Start sample-app and test your changes:

   npm start
   Copy

Appendix

Quick-Start for Adding Eval Metrics

For adding eval metrics, you can skip all the other documents in this repo and quickly get started by following the steps below:

1. Fork this repo and clone it

   git clone https://github.com/<github_username>/palico-ai
   Copy

2. Install npm dependencies

   cd palico-ai
   npm install
   Copy

3. Add your changes for new metric in packages/palico-app/src/experiments/metrics folder. You will need to include the following changes:

   • Add metric definition (example)
   • Add unit tests (example)
   • Export the metric file (example)

4. Confirm project builds properly

   npx nx run sanity-check
   Copy

5. Pull Request