The AI-Powered Owner

AI on Your Own Terms: Free, Open-Source Tools for the AI-Skeptical Owner

By David Heyburn · 2026-10-14 · AI Basics

Illustration of a laptop showing a voice waveform on a kitchen table at night, beside a mug, a phone and a houseplant.
Your AI, your machine. Nothing leaves the kitchen table.

If you feel uneasy about AI, you're not alone, and you're probably not wrong. This is an honest look at the ethical, political and "this is going to end badly" worries many of us share, what each choice actually costs, and how to get the benefits with free, open-source tools if you don't want to fund the companies you're worried about.

I teach small business owners and real estate agents how to use AI as a kind of chief of staff: something that takes the follow-up texts, meeting notes, CRM updates and first drafts off your plate so you can spend your time with people.

So you might expect me to be an AI cheerleader. I'm not.

I don't think AI is an inherent good. I think it's a powerful tool being built very fast, mostly by a small number of very rich companies, with a lot of public money, public data and public trust at stake. When people in my workshops tell me they're uneasy about it, I get it. They aren't confused or behind the times. They're paying attention, and I share a lot of the same feelings.

This post is for them, and honestly, it's for me too. I want to lay out those concerns as fairly as I can, say where I think they're right, compare the options side by side, and then leave the decision with you. That includes the option of not using AI at all.

You don't have to choose between "go all in with Big Tech" and "stay out on principle." There's a lot of room in between, and most people don't know it exists.

What people are worried about, and why it makes sense

When someone tells me AI makes them uncomfortable, it's rarely one thing. It's usually several worries tangled together. Pulling them apart helps, because each one has a different answer, and some don't have a good answer yet. I've felt every one of these myself.

"The data centers are draining power and water."

This one is real. Training and running frontier AI models takes enormous amounts of electricity, and large data centers use a lot of water for cooling. Utilities in several regions have revised their demand forecasts upward because of AI data center growth, and some communities have pushed back against new facilities. Published estimates for any single chat request vary widely and are often disputed, but the total footprint of the industry is large and growing.

Fair? Yes, at the industry level. What helps: a small model on your own laptop uses roughly the power of a video call while it's working and nothing while it's idle. It needs no new data center. The model was still trained somewhere, and that cost is already spent, but using it doesn't fund the next round of construction.

"These models were built by scraping people's work without asking."

Also real. Most large models, open and closed, were trained on huge amounts of text, art, code and writing collected from the internet, mostly without the creators' permission or payment. Several lawsuits from authors, artists, musicians and news organizations are working through the courts, and the legal questions aren't settled.

Fair? Yes, and open-source doesn't fully escape it. Most open-weight models were trained on similarly scraped data. What helps: some projects publish their training data or use more carefully sourced datasets (OLMo from the Allen Institute is one example). You can also limit what you use AI for: drafting your own follow-ups and summarizing your own notes is a very different act from generating "art in the style of" a living illustrator.

"I don't want my clients' information on someone else's servers."

For agents, lenders, therapists, attorneys and anyone else who handles sensitive client data, this matters. With cloud AI, what you type goes to the company's servers. Policies on retention and training vary by company, plan and setting, and they change over time. Business tiers usually offer stronger protections than free consumer accounts, but you're still trusting the provider.

Fair? Yes, especially on free consumer plans. What helps: a local model never sends anything anywhere. This is the cleanest win open-source has: the privacy question goes away.

"I don't trust the people running these companies."

You don't have to. A handful of companies and investors control the most capable models, and their leaders have outsized influence on policy, on information, and increasingly on politics. People across the political spectrum are uneasy about this, for different reasons. Some worry about censorship and ideological slant in what models will and won't say. Others worry about surveillance, military contracts, or billionaires lobbying to write their own rules. Many worry about all of it.

Fair? Reasonable people weigh this differently, but the concentration of power is a fact. What helps: open-weight models can be downloaded and kept. No one can change their behavior, raise their price or cut you off after the fact. You can choose models from different makers, and your subscription dollars are a vote you can withhold.

"AI is going to take people's jobs."

Some jobs, yes, and some of the work it replaces is done by assistants, transaction coordinators and virtual staff, often people in lower-paid roles. It would be dishonest to say otherwise. There's also hidden labor behind AI: content moderators and data labelers, many in lower-wage countries, who review disturbing material to make these systems safer.

Fair? Yes. The effects on work are real and unevenly shared. What helps: no tool choice fixes this. What you control is how you use it. For most solo owners I work with, AI doesn't replace an employee they have. It does work that wasn't getting done, and it gives them room to hire and keep good people. If you have staff, you can use it to make their jobs better instead of shorter.

"It makes things up, and it's flooding the world with junk."

Correct on both counts. Language models can state false things confidently. They make spam, scams, fake reviews and deepfakes cheaper. The internet already has more low-quality AI content than anyone can read.

Fair? Yes. What helps: keep a human in the loop. Use AI to draft, never to send on its own. Never let it invent facts, reviews or credentials. In my own setup the AI marks anything it guessed with [CHECK] so I verify it before a client sees it.

"We're going to forget how to think for ourselves."

This one gets less attention than it deserves. If you hand off every email, every decision and every first thought, your own skills fade. Early research on students and knowledge workers suggests heavy reliance on AI can reduce critical engagement with the work.

Fair? Yes, as a risk. What helps: delegate the clerical, keep the judgment. Let it write the CRM note; you decide what to say to the client who's scared about their inspection report.

"This is going to take over the world."

Some serious researchers, including people who helped build the field, think advanced AI could become dangerous to humanity if it's built carelessly. Others think that fear is overblown and distracts from the harms happening right now. I'm not going to tell you who's right, because I don't know, and neither does anyone else with certainty.

Fair? It's a legitimate debate among people who know the field. What helps: very little of that risk lives in the small model on your laptop writing a follow-up text. If this is your main concern, your leverage is political: support the safety research, regulation and accountability you believe in.

Where I land: most of these concerns are partly or fully right. None of them, for me, adds up to "never touch it." Several of them do add up to "be deliberate about which tools you use, where your data goes, and who you pay."

Your four real options

None of these is the "right" answer for everyone.

  1. Abstain. Don't use AI. Keep doing the work yourself or with people. A completely valid choice.
  2. Commercial cloud. ChatGPT, Claude, Gemini, Copilot. The most capable, the easiest, and the most dependent on big companies.
  3. Fully open and local. Free, open-source tools running on your own computer. Private and independent, with more setup.
  4. Hybrid. Local by default for anything sensitive; commercial only when you choose to, for specific tasks.

How the options compare

Abstain

Commercial cloud

Fully open and local

Hybrid

The open-source chief of staff stack

If option 3 or 4 interests you, this is what it looks like. Every tool below is free, and most are fully open-source. The idea is the same: if you can talk into your phone, you can have a chief of staff. The difference is that it runs on hardware you own. You talk, Whisper transcribes, a local model drafts the text, email and CRM note, n8n routes it, and you review and tap send.

The brain: a local language model

Ollama or LM Studio run models directly on a Mac or PC. Free models include Llama (Meta), Mistral, Qwen, Gemma (Google) and OLMo (Allen Institute). Mid-sized models run well on a modern laptop and handle follow-up texts, email drafts, summaries and notes without trouble. Most of these are "open-weight": you can download, run and keep them, but some licenses add conditions. Mistral's and Qwen's main models use the permissive Apache 2.0 license, and OLMo publishes its training data too.

The interface: a private chat app

Open WebUI gives you a ChatGPT-style window connected to your local model, with saved prompts, document uploads and separate workspaces.

The ears: voice to text

whisper.cpp is excellent speech recognition that runs on your own device under an MIT license. Talk, and it transcribes locally.

The memory: notes and knowledge

Joplin or Logseq keep client notes, scripts and procedures in files you own. Open WebUI can read your documents so answers come from your own information.

The CRM: client relationships

Twenty and EspoCRM are open-source CRMs you can host yourself, with no per-seat fees.

The calendar: scheduling

Cal.com is an open-source, self-hostable booking page for showings, consultations and appointments.

The glue: automation

n8n connects everything and is free to self-host under its "fair-code" license (source-available, with limits on reselling it). This is where a set of tools becomes a chief of staff.

The office: files and documents

Nextcloud and LibreOffice give you private file storage, documents and spreadsheets without a Google or Microsoft account.

The trade-offs: setup takes real time (an afternoon if you're comfortable with computers, a weekend if you're not). An Apple Silicon Mac with 16GB or more of memory works well; older machines will be slow. The biggest commercial models are still better at complex reasoning. And when something breaks, the answer is a forum, not a help desk.

Questions to help you decide

I'm not going to tell you which path to pick. These questions are how I'd think it through.

  1. Which concern weighs on you most? If it's privacy or funding big companies, local is a strong answer. If it's scraped training data, no option fully solves it except abstaining. If it's jobs, your tool choice matters less than how you use it.
  2. What would you actually use it for? Follow-ups, notes and first drafts work fine on local models. Complex analysis is where commercial tools pull ahead.
  3. How sensitive is your data? Client financials, health information or legal matters lean toward local.
  4. How much setup are you willing to do? Be honest. A local system you never finish setting up helps no one.
  5. What's the cost of not doing it? If you're losing leads to slow follow-up or working until 11pm on admin, that has a cost too, to you and your family.
  6. Can you start small? Try one task, like a post-meeting voice debrief, for two weeks before deciding anything bigger.

What I believe

I'm uneasy about how fast this is moving and about who's steering it. I think the creators whose work trained these models deserve better than they've gotten. I think the energy use is a real cost that the industry downplays. I think a world where a few companies sit between us and most of what we read, write and decide is a worse world.

I also watch solo business owners drown in admin every week, and I've seen a simple, private setup give them hours back to spend with clients and family.

Both of those things are true. Open-source tools are one of the few ways I know to act on both at once: use the technology, keep your data, pay no one you don't want to pay, and refuse to make yourself dependent on companies you don't trust. That's how ordinary people have kept technology from belonging only to the powerful before. It's why Linux runs much of the internet and why Wikipedia exists.

If you weigh all this and decide AI isn't for you, I respect that. If you decide to use it, I hope you'll do it deliberately, with your eyes open and on your own terms.

Frequently asked questions

Can I use AI without sending my data to big tech companies?

Yes. Tools like Ollama and LM Studio run open-weight AI models entirely on your own computer, so nothing you type leaves your machine. Pair them with Whisper for voice and Open WebUI for a chat window, and you have a private assistant at no monthly cost.

Are open-source AI models good enough for a small business?

For everyday work like drafting follow-ups, summarizing notes, writing emails and organizing tasks, yes. The largest commercial models are still better at complex reasoning, which is why many owners use a hybrid: local for sensitive work, commercial only when needed.

Is local AI better for the environment?

Running a small model on your laptop uses roughly the power of a video call while it works and adds no new data center demand. The model was still trained in a data center, so it isn't footprint-free, but using it doesn't fund new construction.

Keep reading