Transformative Capacity: DeepSeek HPC Setup, Classification & Explainable AI (Hello World example)

DeepSeek is the only open source LLM on the market that can match the reasoning power of GPT-4o. A major research gap that DeepSeek can address is the classification of data. Classification is a tedious process. There are many areas where it has simply not been possible to develop robust classification schemes that fully account for the desired depth of context and analysis goals.

Two examples from my current research are (1) transformative capacity and (2) social cohesion. Transforming these concepts into operational data filters or classification schemes has not been possible with current natural language processing (NLP). How else would we qualify contextual spatial information to judge whether atomic pieces of information contain aspects that help us assess Transformative Capacity or Social Cohesion?

In a data-rich but information-poor world, this means that we cannot operationalise these concepts to derive insights from the data we have. With the advent of LLMs and the open availability of DeepSeek in particular, this situation has changed.

This is a first glimpse of a research direction I am currently pursuing at the IÖR. The future vision is to provide indicators of transformative change and social cohesion that allow monitoring of these concepts for spatial units and improving decision-making processes in planning.

Until we get there, many technical, conceptual and methodological challenges must be solved.

Practical example

Below is a task I gave deepseek-r1:32b to test classification.

My prompt:

Transformative Capacities refers to a situation where positive change is likely or possible. We want to classify Social Media posts whether they indicate Transformative Capacity. I will give you a social media post text, please respond with 1 (Transformative Capacity likely) or 0 (No Transformative Capacity noticeable) based on the text.

Social Media Post 1:

Would driving into yosemite with a corolla for a day right now be a bad idea, is the park open for public to show up and pay for enterance at gate, and will there be too much snow on roads… I’m from out of state and not sure, thanks!

Answer:

<think>
Okay, so I need to figure out whether the given social media post indicates Transformative 
Capacity. From what I understand, Transformative Capacities refer to situations where 
positive change is likely or possible. So, my task is to analyze the content of the post 
and determine if it suggests any potential for positive change.

Looking at Social Media Post 1: The user is asking about driving into Yosemite with 
a Corolla, checking if it's a bad idea. They're also inquiring whether the park is 
open and if there will be too much snow on the roads. The user mentions they're from 
out of state and unsure about these 
details.

Now, I need to assess if this post shows any signs that something positive might 
change. Let me break down each part:

1. **Driving into Yosemite**: This could imply a desire to visit a natural area, 
which is generally positive. However, the question is whether it's a bad idea. The 
uncertainty here doesn't necessarily
indicate a transformative intention.

2. **Checking park openness and road conditions**: These are practical concerns about 
planning a trip. While visiting Yosemite can be a meaningful experience, just checking 
if the park is accessible doesn't inherently show a likelihood of positive change.

3. **Asking about snow on roads**: This is again a logistical concern. The user wants 
to ensure they can safely make the trip but isn't discussing any plans that would 
lead to transformative outcomes.

Considering all these points, the post seems more like a routine query for travel 
information rather than an indication of potential positive change or transformative 
capacity. There's no mention of personal growth, community impact, environmental efforts, 
or any other factors that could lead to significant positive change. The user is simply 
seeking information about accessibility and safety for a planned trip.

Therefore, I don't see any elements in this post that suggest Transformative Capacity.
</think>

0

I think this is pretty good. The input or task I gave it is very difficult to bring together with the concept of Transformative Capacity, but its reasoning is plausible and the answer makes sense.

How to apply this?

We have public geo-social media data for Germany with about 70 million posts. The hypothesis (to be proven) is that we can use this data to classify regions or areas where transformative change is more likely, based on the type and character of online communication available. The LLM is asked to classify individual posts or users as indicative of TC (instead of 0 or 1, we could ask for a continuum between these numbers). The results are then aggregated using classic GIS methods.

Setup

Until then, here’s a brief summary how I set this up in the TUD High Performance Cluster.

We are using ollama to run deepseek.

First, get the latest release from ollama: https://github.com/ollama/ollama/releases

See the readme for setup instructions

Create a workspace on horse (SSD):

ws_allocate -F horse -r 7 -m <your-s-number>@tu-dresden.de <your-hpc-project> 90

Info: creating workspace.
/data/horse/ws/-
remaining extensions : 10
remaining time in days: 90

cd /data/horse/ws/<your-s-number>-<your-hpc-project>

Download ollama latest release.

curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
tar -C . -xzf ollama-linux-amd64.tgz
rm ollama-linux-amd64.tgz

Add the binary to PATH and test ollama

PATH=$PATH:/data/horse/ws/<your-s-number>-<your-hpc-project>/ollama/bin
ollama serve

If it works, we can continue. In order to run a DeepSeek Model through ollama, serve must be move to a background process. Since this is not possible in the HPC, we can use a jump-host (any VM, or your local computer):

ssh <user>@111.11.11.1 -o ExitOnForwardFailure=yes \
    -o ServerAliveInterval=20 -f -R :11434:127.0.0.1:11434 -p 22 -N; \
    ollama serve

• 111.11.11.1 is the IP of your jump host. It needs SSHD service running and you need to be able to connect to it.
• :11434:127.0.0.1:11434 will bind-forward the port 11434 on both sides
• in other words, ollama run on the remote side can connect to ollama serve in the HPC node

Head over to the remote side (111.11.11.1).

We now need to check requirements and chose the model we can use. With one GPU in the HPC, we have about 40 GB VRAM available.

Check requirements: https://ollama.com/library/deepseek-r1

We need a distilled model. https://apxml.com/posts/gpu-requirements-deepseek-r1

Since we cannot run the full original model into a single GPU-node, we chose the largest quantized model that fits into the available VRAM.

This is DeepSeek-R1-Distill-Qwen-32B (I tested DeepSeek-R1-Distill-Llama-70B and it did not fit).

Extract/install ollama again on the remote host and run it:

./ollama run deepseek-r1:32b

That is it. Below is a screencast, where I ask it to provide information on the IÖR. This is my hello world example.

Question: Can you summarize what you know about the Leibniz-Institut für ökologische Raumentwicklung e.V.?

The Leibniz-Institut für ökologische Raumentwicklung e. V. (Leibniz-IFOR) is a research institution focused on sustainable spatial development and ecological planning. Here’s a summary of what it is about:

1. Mission: The institute works to develop concepts, strategies, and instruments for sustainable land use and urban development that balance economic, social, and environmental goals.
2. Research Focus:
   • Sustainable cities and regions.
   • Ecological spatial development.
   • Regional planning and governance.
   • Climate change adaptation and mitigation in spatial planning.
3. Approach: The institute combines scientific research with practical applications to provide solutions for policymakers, urban planners, and other stakeholders.
4. Role: As part of the Leibniz Association (a network of German research institutions), it contributes to advancing knowledge in sustainable development while addressing real-world challenges.