Our TÜBİTAK 1711 Project, AI-Supported Electronıc Board Qualıty Control Unıt, Has Been Approved!

We were delighted to take part in the 5G Arena at WIN EURASIA for the very first time — made possible through the collaboration of Nokia and Türk Telekom.

It was a valuable opportunity to present our AI-powered industrial solutions through a scenario designed to reflect real-world factory conditions. The insightful feedback we received from visitors and the new collaborations initiated during the event have further strengthened our vision in this field.

We extend our sincere thanks to the WIN EURASIA organizing team and Murat Bayazıt, to Nokia and Türk Telekom for including us in the 5G Arena, and to all public and private sector representatives who visited our booth.
With the inspiration we’ve gained, we continue shaping the future of next-generation industrial environments.

We are proud to announce that our project, carried out in collaboration with ASSAN and Sakarya University within the scope of TÜBİTAK 1711, has been approved!

Our AI-powered project, aimed at transforming quality control processes in electronic board manufacturing, will provide an innovative solution to the industry by detecting production defects with the highest accuracy. This project, supported by TÜBİTAK 1711 and implemented through the collaboration of ASSAN and Sakarya University, stands as a testament to the strong bond between industry and academia.

Through this project, quality control processes in electronic board manufacturing will become faster, more efficient, and more reliable. By leveraging AI technologies, the rate of defective production will be minimized, leading to significant time and cost savings in production processes. At the same time, high-precision analyses will contribute to the production of higher-quality products.

We are deeply honored and proud to bring this project to life, which will contribute to Turkey’s goals in technology and innovation. This initiative, aimed at increasing digitization and the use of advanced technologies in production, will lead the way in raising quality standards across the industry. We extend our gratitude to TÜBİTAK for their support, and to our esteemed partners ASSAN and Sakarya University. We are excited to be part of such a meaningful collaboration, shaping a project that will guide the future. We will continue our efforts with determination to contribute to Turkey’s technology and industrial ecosystem and strengthen our position on the global stage.

We are moving forward without pause for an innovation-driven future!

Clustering methods are a machine learning approach used to group and separate unlabeled data based on specific metrics. These methods come in many different types, depending on their working principles or the similarity measurement techniques they use. But what do labeled and unlabeled data mean?

• Labeled Data: Imagine we want an AI model to distinguish between apples and bananas. When providing the model with apple and banana images as training data, we explicitly indicate which fruit each image belongs to. In other words, the images are labeled as either "apple" or "banana." Additionally, we provide the features of the apple and banana (e.g., color, size, shape, etc.) and show which feature belongs to which object. In this way, the model learns from these labeled data and learns to classify new incoming data accurately.
• Unlabeled Data: Now, let's consider a scenario where we only give AI the features of apples and bananas but don't indicate which features belong to which fruit. So, while the features of apples and bananas are present, we don't specify which feature corresponds to which fruit. In this case, the model analyzes the similarities and differences in the data. It can group similar features together. In this process, clustering methods are used, and the model naturally separates the data into groups

Figure 1 - Supervised and unsupervised learning

How can AI distinguish between apples and bananas without knowing which class the examples in the data belong to? Before answering this question, let's clarify why working with unlabeled data is important.

Working with labeled data is usually more advantageous because we can easily test the model's performance and manipulate the data or model as needed. In labeled data, since we know which class each example belongs to, we can directly evaluate the model's accuracy. However, in real-world problems, we may not always know which class the data belongs to, or labeling the data can be costly and time-consuming.

• Social Media: Users' posts, comments, and interactions are generally collected without labels. Labeling large datasets collected from social media is highly costly because a human needs to read and evaluate each comment individually.

Therefore, we may have to work with unlabeled data. When working with unlabeled data, the model focuses on discovering natural groups or patterns in the examples. By using clustering methods, data with similar features is grouped together. In this way, AI can distinguish different classes, such as apples and bananas, even without class labels.

Let's take a look at some popular clustering methods:

K-means Clustering Algorithm

• Why is it used?
- Data Grouping: It is used to group similar items in a dataset, making data analysis and interpretation easier.
- Feature Selection: It helps identify important features in a data set.
- Image Processing: It is used to group images based on similar features, such as grouping colors (see Figure 2).

Figure 2 - Image segmented with K-means

 
Figure 3 - Clustering result using K-means with the sklearn make_moons dataset

 
Figure 4 - Clustering result using K-means on the Iris dataset

As seen in Figure 3 and Figure 4, the K-means method does not provide the desired performance because it calculates clusters circularly.

DBSCAN Clustering Algorithm

• Why is it used?
- Density-Based Grouping: It is used to identify dense regions in the data. It groups data by focusing on dense regions.
- Outlier Detection: It is effective at identifying outliers (noise); points in less dense regions are considered outliers (see Figure 5).
- Distinct Clusters: It allows for determining cluster shapes without being bound to any geometric structure, enabling the detection of complex and irregularly shaped clusters (see Figure 6).

Figure 5 - Clustering result using DBSCAN on the Iris dataset

 
Figure 6 - Clustering result using DBSCAN on the sklearn make_moons dataset

Gaussian Mixture Model

• Why is it used?
- Complex Distributions: Unlike other clustering methods, GMM can be preferred in complex data structures because it uses Gaussian probability distribution for each cluster (see Figure 7).

 
Figure 7 - Clustering result using GMM on the Iris dataset
 
Figure 8 - Clustering result using GMM on the sklearn make_moons dataset

Evaluating and Improving Clustering Results

Evaluating the performance of clustering algorithms is a critical step in obtaining accurate results. However, since clustering is an unsupervised learning method, it is difficult to directly measure the results when labeled data is absent. Nevertheless, the quality of the clusters can be evaluated using various metrics:

• Silhouette Score: It is calculated by comparing each example’s distance from its own cluster to its distance from other clusters. A high score indicates that the clusters are well-separated and tightly grouped.
• Davies-Bouldin Index: This metric measures the similarity between clusters by considering the distances between cluster centers. A lower Davies-Bouldin score indicates well-separated clusters.
• Rand Index: It evaluates the quality of clusters by measuring similarities and differences in the data.

While evaluating clustering results with such metrics, the performance of algorithms can also be improved through optimization. For instance, by tuning hyperparameters or using dimensionality reduction techniques, clustering algorithm outputs can be enhanced. Adjustments to the parameters provided to the algorithm and data features can help achieve more accurate and meaningful groupings.

Outlier Detection with Clustering

Clustering algorithms not only bring similar data together but can also be used to detect outliers. Density-based algorithms like DBSCAN are particularly effective in this area. Points outside the clusters formed by dense data points can be classified as outliers. These outliers often represent data that is unnatural, incorrectly measured, or extreme events in the dataset. Detecting outliers plays a vital role in many fields:

• Finance: In fraud detection, abnormal transactions that deviate from normal transaction clusters can be identified.
• Healthcare: In patients' normal health indicators, abnormal and risky conditions can be detected as outliers.
• Industrial Sensor Data: In data recorded by sensors on production lines, outliers can help identify machine malfunctions or faulty measurements.

Clustering algorithms are widely used across different industries and application areas. For example, in marketing, companies use clustering algorithms to segment their customers based on behavior and demographic features. By understanding the target audience better, they can develop marketing strategies and improve customer satisfaction. In bioinformatics, clustering algorithms play a significant role in analyzing genetic data. Clustering genome data allows for discovering relationships between genes and proteins, helping us better understand biological processes.

Clustering methods are also frequently used in image processing. The K-means algorithm, in particular, is widely preferred in processes such as image segmentation and separating color clusters. Another important application area is social media analysis. The vast amounts of data collected from social media are analyzed using clustering algorithms to examine user behavior, identify trends, and create personalized content recommendations. All these examples show that clustering algorithms play an important role in data analysis and have a wide range of applications.

Conclusion

When we want to use AI in everyday life, several problems arise:

• Data is unlabeled: Labeling data often requires the help of an expert. However, even an expert in the field may not be able to label the data 100% accurately.
• The data is vast and complex: Working with large and complex data makes operations more challenging, requiring more time and resources.

For these reasons, unsupervised learning methods play an important role in solving real-life problems. In this article, we briefly discussed clustering methods, one of the unsupervised learning methods. Besides methods like K-means, DBSCAN, and GMM (Gaussian Mixture Model), many different clustering techniques exist. Depending on the problem at hand, the advantages and disadvantages of these methods should be evaluated, and the most suitable one should be chosen. Simple but effective clustering methods still assist developers in solving fundamental problems today.

References

https://www.researchgate.net/figure/Supervised-learning-and-unsupervised-learning-Supervised-learning-uses-annotation_fig1_329533120
https://www.freecodecamp.org/news/8-clustering-algorithms-in-machine-learning-that-all-data-scientists-should-know/
https://www.geeksforgeeks.org/clustering-based-approaches-for-outlier-detection-in-data-mining/

Remember the Biology classes you took in high school. Carbon atoms, which are the basis of life, combine with certain other elements to form organic compounds. These organic compounds even include the medicines prescribed to us by our doctor when we are sick. So can we find a pattern between these molecules? Or can artificial intelligence design the treatment drugs for new diseases in the future?

Figure 1 : Methyldopamine Compound⁽¹⁾ (Used in the Treatment of Parkinson's Disease)

We will answer these questions in our article. But first, we will talk about a new algorithm that has recently gained a lot of popularity in the artificial intelligence sector. Graph Neural Networks, or GNNs.

What are Graph Neural Networks (GNNs)?

In the early 2000s, data scientists were struggling to model complex relationships with traditional methods. Social networks, molecules and even texts. Each was like a network of dots and connecting lines. But it was not easy to uncover hidden relationships and patterns in these networks. This is where GNNs came on the scene.

GNNs are basically a special deep learning algorithm designed to understand and analyze relationships. Many things in the data world can actually be represented by interconnected nodes and edges between these nodes. GNNs work on such “graphs” to learn the complex relationships between nodes and edges and extract meaningful information.

Figure 2 : A Graph Example⁽²⁾

Like a detective, these algorithms follow the connections in the data. Each dot represents a piece of information and each line represents a relationship. By tracing these connections, GNNs try to uncover the secrets deep in the data.

The early days of GNNs, like other deep learning algorithms, were quite challenging. GNNs were still in their infancy and struggled to deal with complex problems. But scientists didn't give up and developed new ideas and algorithms. GNNs became stronger over time and started to be quite successful in solving their own problems. The 2010s were a turning point for GNNs. With the explosion of social media, massive social networks emerged. GNNs revolutionized the detection of friendships, communities and even fake accounts on these networks.

Drug discovery was another area where GNNs shone. Understanding the complex structure of molecules was critical for developing new drugs. By modeling the interactions between molecules, GNNs helped scientists find new drug candidates. Nowadays, GNNs have become an indispensable tool in data science. They are used in many areas from recommendation systems to traffic forecasting, natural language processing to text analysis.

So, Why Do We Need GNNs?

Why do we use GNNs when there are Transformer algorithms or many other algorithms that remain very popular today? Of course, there are many answers to this question. But one is much more important than the other answers, data structures.

Traditional AI models [e.g. convolutional neural networks (CNNs)] usually process data in regular structures (lists, tables, images). But a lot of real-world data is organized into networks (graphs). For example, your network of contacts on social media, the highway network you use while traveling in your car, the recommendation systems used in shopping websites, the relationships between words and conjunctions in the language you use, and the relationships of networks between molecules we have already described; when we want to store such data in our digital environments, the most optimal data structures we apply will be graphs. GNNs, on the other hand, can directly process the data in such graph structures and learn the complex relationships within them, establishing new, previously undiscovered relationships between different networks.

Figure 3 : Differences in the Data Types of CNN and GNN Algorithms⁽³⁾

What are the Basic Building Blocks of GNNs?

• Graph: GNNs work on a graph data structure. A graph consists of a set of nodes and a set of edges. Nodes represent entities in the data (e.g. people, molecules, words), while edges represent relationships between these entities (e.g. friendships, chemical bonds, semantic relationships).
• Node Features: Each node can have a set of features (e.g. age, atomic number, word vector) that describe it. These features help represent the nodes and are processed by the GNN.
• Edge Features: In some graph structures, edges can also have properties that describe them (e.g. type, weight, direction of the relationship). These properties allow for a more detailed modeling of the relationships between nodes.

Working Stages of GNNs

GNNs are usually trained by supervised learning or semi-supervised learning methods. During the training process, the parameters of the model are optimized using graph structure and node/edge features. The goal is to maximize the model's performance on a given task.

GNNs basically work in the following steps:

• Message Passing:

It allows each node to enrich its representation (feature vector) by receiving messages (information) from other neighboring nodes. This information is computed from the existing representations of the neighboring nodes and the properties of the edges between them. The information computation function here is usually a neural network with learnable parameters [e.g. a multilayer perceptron (MLP)].

• Aggregation:

It allows each node to combine all the messages it receives into a single representation. The aggregation function used here is usually executed using averaging, summing, maximization or attention mechanism operations. This step helps the node to summarize information from its neighbors.

• Update:

It allows each node to update its representation with a new one, using the messages it has collected and its current representation. This function is usually a neural network with learnable parameters (e.g. MLP, RNN).

• Iteration:

It ensures that the message passing, aggregation and update steps are repeated (usually several layers or iterations) until information is propagated throughout the entire graph structure. With each iteration, nodes collect more information from their neighbors and their representations become richer. This iteration process allows GNNs to capture distant dependencies in the graph structure.

• Output:

After its last iteration, node representations can be used for various tasks such as node classification, link prediction or graph classification. The output layer is usually a neural network with learnable parameters and is designed in a task-specific way.

Conclusion

To answer the question we asked you at the beginning of this article, at the moment, no. But we do know that they play an important role in the drug discovery process and help identify new drug candidates. When we think about it, an AI that can make sense of the bonds between molecules in the same way that transformer algorithms can make sense of words (thanks to word embedding) doesn't sound too futuristic, does it?

Graph Neural Networks (GNNs) have the groundbreaking potential to make sense of complex relationships in the data world and extract valuable information from these relationships. GNNs offer a powerful solution in many areas where traditional methods fall short, such as social network analysis, recommender systems, drug discovery, traffic prediction and natural language processing. The successful applications of GNNs in these areas make their future potential even more exciting.

The ability of GNNs to handle larger and more complex data sets, to provide explainability and transparency, and to model dynamic and time-varying graphs will open new horizons in the field of artificial intelligence. The integration of GNNs with other artificial intelligence techniques will enable the development of more powerful and versatile applications.

GNNs are a discovery that continues to push the boundaries of human intelligence and curiosity. And we are witnessing the first moments of these discoveries.

References:

1. α-Methyldoppamine : https://en.wikipedia.org/wiki/%CE%91-Methyldopamine
2. Graphs : https://adacomputerscience.org/concepts/struct_graph?examBoard=ada&stage=all
3. Graph Neural Network and Some of GNN Applications: Everything You Need to Know : https://neptune.ai/blog/graph-neural-network-and-some-of-gnn-applications

Large Language Models (LLM) are one of the most important technologies today, and as the name suggests, they are truly big. But this does not mean that they are perfect. Sometimes their answers to the questions you ask them make it seem like they don't know what they're talking about, because they really don't know anything. Large Language Models focus on the statistical relationships between words, rather than understanding their meanings. This causes hallucinations, which is one of the most important problems of LLMs.

Another problem is that a Large Language Model becomes outdated even shortly after it is trained. We cannot re-train them every moment after a new event or new discovery. Because this is a very costly and inefficient approach. In other words, they are frozen in time and are like a closed box. We don't know where they get the information they give anyway. It would be like if we ask a person which planet has the most moons in our solar system and he answers, "I saw in a magazine 10 years ago when I was a kid that Jupiter had 88 moons." We do not know the source of this information and it is out of date.

And if we want our Large Language Model to work with very high accuracy and efficiency in a domain-specific way, the currently available LLMs will not meet this request. Because they are trained on broad and diverse datasets meant to cover a wide range of topics, rather than being specialized in a particular domain. As a solution to this, we can fine-tune them with data sets specific to this field. Although this method is a costly task, it provides the Large Language Model with a general perspective specific to that field. However, considering that these models are trained with very large data sets beforehand, the data set we use for fine-tuning will not be enough to make it perfect at a task. In order to reduce or solve all these problems, Facebook Research introduced the Retrieval Augmented Generation (RAG) method in 2020. This method has a much lower cost than fine tuning and is easier to implement. RAG has multiple methods, but the general logic is the same in all of them. A query is obtained according to the entered request. Then, using this query, the most relevant data is retrieved from predetermined information sources. These fetched data is combined with the request and presented to the Large Language Model. This ensures that the LLM has up-to-date or contextual information.

What are Query and Prompt?

We will see these two concepts quite often in the rest of the article. That's why it would be good to make the distinction at the beginning. To get a response from the Large Language Model, the user have to ask it some questions or enter instructions. This entire input is called the prompt. Even a small word or letter change and the answer can sound completely different. The practice of finding the most suitable prompt is called prompt engineering. The query comes into play when we include RAG in the system. In this case, we would like to give the Large Language Model a context other than our own prompt. This context may be up-to-date data that LLM does not have on the subject for which the answer is expected, or it may be company-specific data. Maybe we would like to bring the most relevant data from thousands of different information sources and add it to the prompt. In this case, we build a query to scan information sources with different algorithms. In some cases, this may be directly the same as the claim.

How Doees RAG Work?

Although the RAG system can be defined as many smaller modules, it basically has two main stages. First, the Retrieval phase and the second is the generation phase. In the first stage, using the query we built before, it finds and retrieves the ones that are semantically or mathematically most similar to the query among single or multiple information sources. These information sources can be web search results, company documents, or a database.

Figure-1: A Fundamental RAG Architecture

The reason why the second stage is called the generation stage is that we include the context or contexts we obtained in the first stage into the request according to a determined format. This generated new prompt is given to the Large Language Model. And now the Large Language Model produces more consistent and up-to-date information while adhering to context.

RAG Stages

As I mentioned in the previous title, a RAG system consists of smaller submodules under two basic modules: query creation, query translation, routing, indexing, retrieval and generation modules. I will briefly touch upon these modules one by one.

Figure-2: RAG Modules diagram

Query Construction

The first stage in RAG systems is query generation. The natural language query obtained from the human-generated prompt is transformed into a format that different data sources can understand. This can be SQL queries for SQL-based databases or word embeddings in vector databases. This part is important because we need to make sure that the query is interpreted correctly in order to obtain the information from the database in the most optimal way.

Query Translation

The query translation phase involves making a natural language query more understandable by breaking it into smaller, more specific parts. This stage basically includes two important steps. The first step is known as Query Decomposition, where the original query is decomposed or reformulated into subqueries. In this way, the more complex and versatile structure of the query is broken down into smaller and more manageable components. Techniques used during this process include methods such as Multi-query, Step-back and RAG-Fusion.

The second step is the creation of Pseudo-documents. In this process, hypothesis documents representing possible answers are created in response to the query. These documents can be thought of as documents that do not actually exist but contain potentially relevant information. The HyDE method used here enables the creation of such documents. As a result, the Query Translation phase involves various transformation and parsing operations to be able to process a query more effectively and ultimately find the correct answer.

Query Generation: Various subqueries are derived from the query entered by the user. These subqueries attempt to fully understand the user's intent by capturing different perspectives. This ensures that the query is addressed more comprehensively.

Sub-query Retrieval: For each sub-query, relevant information is collected from large data sets and repositories. This step is done to obtain comprehensive and in-depth search results.

Reciprocal Rank Fusion: The retrieved documents are combined using the Reciprocal Rank Fusion (RRF) method. This method prioritizes the most relevant and comprehensive results by combining sequences of documents, thus allowing us to get the best answers.

Routing

The Routing stage allows deciding which database or information source the incoming query will be directed to when there is more than one data source. This can be done in two different ways depending on how it works: logical routing and semantic routing.

Figure-3: Routing Example

Logical Routing: In this method, it refers to deciding which of multiple data sources to use, depending on the structure of the query, as a result of a logical evaluation. At this stage, if desired, the Large Language Model can be used for this selection process.

Semantic Routing: In this routing method, it is decided which of the answers from multiple data sources will be more appropriate, based on the meaning of the generated query rather than logically. To achieve this, word embeddings can be used, because vectors have direction and positional meaning in a space.

I explained the remaining stages with the definition I made in the introduction. To briefly summarize, the Retrieval phase involves retrieving and listing the relevant documents after selecting the data source. Different ranking algorithms may work in this process. In the indexing phase, the data is first divided into chunks and organized, stored in different formats, processed with field-specific embedding techniques, and document summaries are grouped according to different levels in a tree structure. In the last stage, generation, the response is obtained by using drawn documents and rewritten requests to produce the most appropriate response to the request.

Reciprocal Rank Fusion (RRF)

Reciprocal Rank Fusion (RRF) is an algorithm used to combine multiple search results to create a single ranked result set. RRF works by assigning a reverse ranking score to the ranking of each search result. These scores are then summed and the results are ranked based on these combined scores. It is used in situations where multiple queries are run in parallel, especially in scenarios such as hash search and multiple vector queries. RRF gives higher importance to items that rank higher in multiple lists, taking into account the original ordering of items in each result list. This improves the overall quality and reliability of the final ranking. An example can be given as follows: When you search for the same thing on the Internet, different search engines or methods present you results with different rankings. RRF takes the results from these different rankings and combines them all, deeming the ones at the top more important. This way, the most relevant results appear at the top of the newly created list. This method is used to best sort results from different sources into a single list.

Creating Simple RAG Using LangChain

LangChain is a tool that allows you to create a chain or flow to perform various tasks using Large Language Models and other AI tools. In particular, it provides tools to use LLMs more efficiently and targetedly in tasks such as natural language processing and text generation. LangChain is used in applications such as data querying, document searching, summarization and decision support systems, and is often used to integrate LLMs into more complex workflows. It can be used for many systems, from simple to complex. I have added a simple code example below from the LangChain documentation to create the RAG system.

Figure-4: RAG creation code with LangChain

Conclusion

The fact that LLMs become outdated over time and can cause problems such as hallucinations are among the main problems these models face in practical applications. RAG is a method that emerged to provide solutions to these problems and stands out as a lower-cost alternative to fine-tuning. The operation of RAG is explained in detail through various stages and modules, with particular emphasis on query construction, translation and generation stages. Thanks to tools like LangChain, RAG systems can be created and integrated simply. These studies represent an important step towards using LLMs more efficiently and accurately.

References

1- Amazon Web Services. What is RAG (Retrieval-Augmented Generation?)
2- Yöndem, D. Retrieval Augmented Generation'a Giriş [Video].
3- IBM Research. What is retrieval-augmented generation?
4- A Comprehensive Guide to RAG Implementations. NoCode.ai Newsletter.
5- Singh, S. Mastering RAG: Advanced Methods to Enhance Retrieval-Augmented Generation.
6- Build a Retrieval Augmented Generation (RAG) App, LangChain Dökümantasyonu
7- Relevance scoring in hybrid search using Reciprocal Rank Fusion (RRF), Microsoft

Generative Adversarial Networks (GANs), one of the most exciting and innovative areas in artificial intelligence, is a unique learning method based on the competition between two neural network models. One of these models is called the "generator," and the other is the "discriminator." The generator aims to produce realistic data, while the discriminator tries to distinguish whether the data is real or fake. Through this competitive process, GANs can generate highly realistic outputs, such as images, sounds, and more.

Developed in 2014 by Ian Goodfellow and his team, this technology has revolutionized the world of artificial intelligence and machine learning. Our company is leveraging the GAN architecture, one of the most innovative approaches in this field, to advance our projects.

In this article, we will explore what GANs are, how they work, and why they are so significant. As we delve into the exciting world of GAN technology, we aim to open the doors to a different realm for you.

Figure 1: Applications of GANs

What is GAN?

Generative Adversarial Networks (GANs) are a deep learning architecture that trains two neural networks to compete with each other in order to generate more unique new data from a given training dataset. For example, you can produce new images from an existing image database or create original music from a song database. GANs are called "adversarial" because they involve training two different networks and pitting them against each other. One network generates new data by altering input data samples as much as possible. The other network tries to predict whether the generated data output belongs to the original dataset. In other words, the predicting network determines whether the generated data is fake or real. The system continues to produce newer and improved versions of fake data until the predicting network can no longer distinguish between fake and real data.

Basic Structure of GANs

GANs have two main components: the Generator and the Discriminator. The Generator processes random input data to produce realistic visuals or data. The Discriminator receives both real data and fake data generated by the Generator, and it tries to distinguish between the real and fake data. These two components interact with each other and continuously improve. The Generator strives to produce better and more realistic data so that the Discriminator cannot accurately detect the fake data, while the Discriminator works to improve its ability to identify fake data. This competitive process enables both components to become more capable over time.

  
Figure 2: Basic GAN Architecture

How GANs Work?

Generative Adversarial Network (GAN) is a system composed of two deep neural networks. One side is the Generator, and the other side is the Discriminator. This system is based on an adversarial scenario where the Generator attempts to produce new data, and the Discriminator tries to determine whether the produced data is real or fake.

The working principle of GANs can be summarized with the following steps:

1. The Generator neural network analyzes the training set and identifies data features.
2. The Discriminator neural network analyzes the initial training data and makes distinctions between features independently.
3. The Generator network modifies some data features by adding noise to specific attributes.
4. The modified data is sent to the Discriminator network, which calculates the likelihood that the generated output belongs to the original dataset.
5. The Discriminator network, in the next iteration, guides the Generator to reduce the randomization of the noise vector.
6. The Generator tries to maximize the Discriminator’s error probability, while the Discriminator aims to minimize the error probability.
7. Throughout the training iterations, both the Generator and Discriminator continuously undergo changes and develop adversarially.
8. When the training process reaches equilibrium, the Discriminator can no longer distinguish synthesized data, and training concludes.

What are the types of GANs?

1. Standard GAN (SGAN): The basic GAN structure introduced by Ian Goodfellow and colleagues in 2014. It is the version where the Generator and Discriminator have a basic adversarial setup.
2. Conditional GAN (CGAN): The version of GAN where the Generator and Discriminator are trained based on a specific condition (usually label or class information). This method is useful for synthesizing or transforming examples with certain features.
3. Deep Convolutional GAN (DCGAN): A type of GAN enhanced with convolutional neural networks (CNNs). It focuses on producing more stable and high-quality results for image data.
4. Wasserstein GAN (WGAN): Developed to address some issues encountered in the training process of GANs. It uses the Wasserstein distance metric to improve training stability and prevent gradient problems.
5. Progressive GAN: Proposed by NVIDIA, this GAN type trains to produce high-resolution images progressively. It starts with low-resolution and gradually enlarges the network to obtain high-resolution images.
6. StyleGAN and StyleGAN2: Developed by NVIDIA, these GAN types use style transfer techniques for high-quality and high-resolution image synthesis. They are especially successful for face and human image generation.
7. CycleGAN: Used for translating between different domains. For example, it can be used for converting images or videos from one style to another.
8. Self-Attention GAN (SAGAN): Uses attention mechanisms to handle long-range dependencies and provides better results for larger and more complex datasets.

 
Figure-3 Basic Types of GANs

In this article, we will discuss the Style GAN type.

What is Style GAN?

Style GAN (Style Generative Adversarial Network) is a type of GAN developed by NVIDIA researchers. Unlike traditional GANs, Style GAN is distinguished by its ability to separately manage the style and content of generated images. The Generator network takes random vectors and style vectors (latent space) as input. Style vectors provide direct control over the colors, textures, and other visual attributes of the generated image. This allows Style GAN to enhance visual quality and produce more realistic and diverse images.

Figure-4 Photos generated with Style GAN

What is Latent Space and How is it Used in Style GAN?

Latent Space is an abstract space that represents the features of an image. In Style GAN, this space is represented by random vectors (latent vectors). These vectors are used to determine the style or characteristics of the image. For example, a latent vector can control features such as hair color, facial expression, or background.

Mapping Network and Synthesis Network Structure

In Style GAN, there are two main network structures:

1. Mapping Network: This network takes random latent vectors and transforms them into a space suitable for image synthesis. The Mapping Network ensures that latent vectors are distributed in a more organized manner, making image synthesis more consistent and controllable.
2. Synthesis Network: This network takes latent vectors produced by the Mapping Network and transforms them into realistic images. The Synthesis Network improves the details and structural features of the image at each layer, resulting in more realistic and detailed images.

These structures form the foundation for extending and optimizing Style GAN. Developers can use these structures to create unique and innovative visuals, making Style GAN applicable in various fields such as art, fashion, and digital content creation.

Applications of GAN and Style GAN

1. Art and Innovative Industries: Style GAN allows artists to create innovative digital artworks and achieve high-resolution, realistic visuals.
2. Fashion Design: In the fashion industry, GANs and Style GANs are used to create new patterns, fabric textures, and clothing styles. This method helps fashion designers develop new trends and collections without limiting their innovation capabilities.
3. Game Development: GAN technologies assist game developers in creating game worlds and characters, providing more realistic and diverse visuals.
4. Medicine and Healthcare:

• • Medical Imaging: GANs can enhance or recreate images in medical imaging techniques such as MRI and CT scans, aiding doctors in making accurate diagnoses.
  • Surgical Simulations: GANs can be used for simulating complex surgical procedures, offering practical training opportunities in surgical education.

5. Machinery and Automotive Industry: GANs can improve the design of new parts and components in product design and optimization processes, contributing to innovation in the machinery and automotive industries.
6. Education and Simulation: GANs can be used to create educational simulations and virtual lab environments, allowing students to practice and apply theoretical knowledge.

References
1. https://www.innodeed.com/wp-content/uploads/2022/09/GAN-1170x640.png
2. https://cdn1.ntv.com.tr/gorsel/IKHMHEUbe0et1mQm8yWUUg.jpg?width=1000&mode=both&scale=both&v=1545216985506
3. Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm. https://www.mdpi.com/2213968
4. https://towardsdatascience.com/generative-adversarial-networks-gans-a-beginners-guide-f37c9f3b7817
5. Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art. https://www.researchgate.net/publication/373551906_Ten_Years_of_Generative_Adversarial_Nets_GANs_A_survey_of_the_state-of-the-art.
6. GAN nedir? - Çekişmeli Üretici Ağlara Ayrıntılı Bakış - AWS (amazon.com)
7. Çekişmeli Üretici Ağlar (GAN). Çekişmeli üretici ağlar, 2014 yılında… | by Burcu Koca | Deep Learning Türkiye | Medium
8. 942990 (dergipark.org.tr)

In this blog post, we will talk a little about Large Language Models that make their impact felt in a wide range of areas, from primary school children to many professionals working in technical fields. However, we may all ask the question of how this vast subject can be summarized. Our aim here is to give the perspective of someone taking a stroll by the seaside by listening to the sounds of seagulls. In addition, by briefly mentioning its historical period, its usage areas, working mechanism, some technical terms and a sample code will be given.

What are Large Language Models and What Are Their Uses?
Natural Language Processing is the field of artificial intelligence used to perform language-related tasks and is already present in our lives. These systems could perform the desired tasks at a certain level, but they were not good enough. The article "Attention is All You Need", published by the Google Research team in 2017 with the aim of improving language translation, opened the door for the development of Large Language Models that can perform these tasks much better. Later, with increasing processing power and textual data, BERT, also developed by Google and using the Attention mechanism, and OpenAI's GPT models with high parameter numbers and unsupervised pre-training continued this development.

Large Language Models are advanced deep learning models with a very large number of parameters, trained on textual data obtained from different sources such as the internet, books, articles and video transcripts. Trained on a wide variety of Natural Language Processing (NLP) tasks, these advanced models essentially predict the next word based on the textual input they receive.

The application areas of Large Language Models cover a much wider range. The ability to interact with people is most evident in chatbots that carry out natural and meaningful conversations. These bots can be used in customer service, entertainment chats, and educational materials. Using Large Language Models, it is possible to create different types of content such as advertisements, blog posts, social media posts. By training specific language models on a particular topic, the quality, fluency and originality of the content can be increased. Large Language Models that analyze the emotional tone (positive, negative, neutral) of texts and the topic covered are useful in measuring customer satisfaction, tracking social media trends, and even evaluating the effectiveness of advertising campaigns.

Large Language Models that produce summaries of long texts contain important information and reflect the essence of the original text. It is possible to use this feature usefully in academic research and in the creation of summaries and reports. Large Language Models are also capable of generating code snippets in programming languages . This ability can make coding processes easier, reduce error rates, and allow people without coding skills to develop software.

In addition, major language models are very successful in translating between different languages.

How do Large Language Models work?

Tokenization
Computers do not perceive data in the same way as we do; they only see numbers (1 or 0). Therefore, the textual data given to the AI model during training or during execution must be expressed in numbers. For this purpose, texts are first divided into tokens. Tokens are the division of large textual data into smaller, meaningful pieces. This process is called tokenization. For example, the tokenizer used by OpenAI divides the text “ILGE Artificial Intelligence” into 6 different tokens as follows.

Word Embeddings
The data, which was tokenized in the previous stage, is converted into mathematically meaningful vectors in this stage. These vectors not only represent words numerically, with factors such as proximity and angle between them, but also represent the relationships between each other. These stages are one of the basic steps in the field of language processing and were introduced to this field long before Large Language Models. While vectorizing words, deep learning techniques are used to learn their context with each other, filling in the gaps in the sentence, predicting other words around the word, and similar tasks. Thanks to this learning, words that are semantically close to each other are located in closer vectors, while words with antonyms are located in opposite positions.

Positional Encoding
Large Language Models must take into account not only the meanings of words but also the order of words when processing word sequences. However, word vectors (embeddings) do not carry sequence information on their own. For example, "Ali went to school." and "Ali went to school." Although the sentences contain the same words, they are different sentences. Therefore, positional coding is used for models to understand such order differences.

 
Figure-1 An example positional coding

Positional coding clarifies the sequential place of the token in the sentence by expressing the position of each token with a mathematical vector. This is often done using sine and cosine functions, as these functions are periodic, providing unique yet scalable encodings at different positions.

Figure-2 Sine/cosine functions for positional coding

Self Attention Mechanism
For us humans, “The pizza came out of the oven and it tasted good!” It is easy to understand that what tastes good is pizza, not the oven, but the same is not true for machines. Self-attention is the building block mechanism of the Transformer architecture , which enables longer sequences to be processed meaningfully and makes Large Language Models so powerful. Each of the vectors in the sequence evaluates its relationships with other vectors and determines the importance and context of each word in the sentence.

 
Figure-3 Transformer model architecture

In the self-attention mechanism, key , value and query concepts are used to model the relationships between data. These values are obtained by multiplying the vector with a set of weights learned during training. In this way, a query vector is created for each word. This vector is used to determine relationships with other words. A key vector is then generated for each word. These keys are compared with queries of other words to determine the degree of influence each word has on each other. As for value, when keys compatible with the query are found, the values connected to these keys are added according to their importance and create a new, updated representation of the word.

For each word in a sentence, the self-attention mechanism calculates the relationship of this word to other words. The query vector of each word is multiplied by the key vectors of all other words to calculate a set of scores (attention score). These scores are normalized with the softmax function, so that the effect of each word on other words is expressed as a probability distribution.

The normalized scores are then multiplied by the relevant value vectors and the results are summed to create a new vector (attention output) for each word. These new vectors are enriched representations that better reflect the meaning and context of words within the sentence.

Multi-Head Attention
Multi-head attention, which is an important part of the Transformer architecture, occurs by using more than one attention head together. To understand the working logic of this structure, filters in CNNs can be considered. Different filters capture different features in the image. The situation is similar in multi-head attention. Different parts of the sentence are focused on by using many attention heads with different weights. These heads learn and detect relationships in different parts of the sentence.

Figure-4 Structure of the multi-head attention layer

Masked Multi-Head Attention
Masked multi-head attention is particularly important in language modelling. This mechanism allows the model to predict the next word just by looking at the previous words. Masked multi-head attention masks future words, preventing the model from accessing future information. The purpose of this is to make it possible to make predictions using only the previous words at each step when building a language model. For example, when generating a sentence word by word, the model does not need to know the words ahead, and this mechanism ensures this.

After the attention outputs are obtained, the Add & Norm layer includes two basic operations. Residual Connection adds the original input to the input, allowing gradients to be backpropagated more effectively in deep models and making it easier for deeper layers to learn. Layer Normalization normalizes each component of the input. As a result, it enables the model to learn faster and more stable.

The add and normalize layer, the feed forward layer comes into play. The feedforward layer is a fully connected neural network that operates independently for each position and usually consists of two linear transformations and an activation function. This layer allows the model to learn more complex and abstract levels of representation. The output from the feedforward layer is sent back to the add and normalize layer and then passes to the linear layer.

Finally, Linear and Softmax layers are applied. The linear layer transforms the model's output into a form suitable for classification. The Softmax layer normalizes the model's predictions for each probability and the probability distribution is obtained. At the end of these stages, the Large Language Model has produced the most probable word or sequence of words based on the input.

Fine-Tuning the Large Language Model
When starting a project, we first determine our problem. After understanding the problem well, we can conclude that the best solution is Large Language Models. We are aware of how difficult it is to obtain enough data and computing power to produce and train a Large Language Model from scratch. Therefore, the first thing we need to do is to test open source or closed source language models on our problem. We can run an open source model on our own system or on a cloud service. We include closed source systems in our projects by paying a usage fee. 
Before considering fine-tuning, we need to optimize the system prompt; This is called prompt engineering. If after this situation we are still far from the results we want, we should look for and try Large Language Models that have already been fine-tuned. For example, we can access such models from sites such as Huggingface, Kaggle etc. If there is still no positive result after all these stages, we choose the fine-tuning method suitable for our problem, collect the appropriate data and start training for fine-tuning. Currently, fine-tuning with LoRA and QLoRA is very popular for performance and convenience.

Fine Tuning with LoRA
You can check out the code shared by a Kaggle user on how to fine-tune with LoRA on a model that is relatively simple and easier to train.
 

Conclusion:
Large Language Models are groundbreaking artificial intelligence systems in the field of Natural Language Processing. By training on large datasets, these models can perform human-like text generation, translation, summarization, and many more language-based tasks. The basic building blocks of LLMs are techniques such as tokenization, word embeddings, positional coding and self-attention mechanism. In particular, the Transformer architecture developed by Google in 2017 has provided a huge leap in the performance of these models. Transformers' multi-head attention mechanism plays a critical role in understanding the complex structure of language and correctly processing the context of texts.
LLMs are used in a wide variety of areas, from customer service to content creation and programming. Fine-tuning techniques allow these models to be better suited for specific tasks. Methods such as LoRA (Low-Rank Adaptation) increase the fine-tuning performance of existing models, contributing to the efficiency of the models and allowing them to be used in more specific tasks.
As a result, Large Language Models are providing revolutionary advances in many areas of NLP and paving the way for broader and more innovative applications in the future. The development and applicability of these models makes it possible to achieve human-like performance in language processing tasks.

References
1- A. Vaswani et al., "Attention is all you need," 2017
2- Raschka, S. (2023, February 9). Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch.
3- Abdin, A. (2023). How to Fine-tune LLMs with LoRA. https://www.kaggle.com/code/aliabdin1/how-to-finetune-llms-with-lora
4- StatQuest with Josh Starmer, Transformer Neural Networks, ChatGPT's foundation, Clearly Explained

We participated in the Sectoral Applications session held under the main theme of "Artificial Intelligence and Entrepreneurship" at the Gebze Technical University (GTU) Entrepreneurship Summit 2024, which took place on May 22nd. The summit brought together digital entrepreneurs, leading companies in the sector, and young entrepreneurs graduated from GTU.

At the event, significant discussions were held on the critical impacts of indigenous and innovative solutions on individuals, businesses, economies, and societies, and how these impacts shape the entrepreneurship ecosystem.

We sincerely thank Gebze Technical University for the invitation to this meaningful event.

In the era of Industry 4.0, automation and efficiency in production processes have become more important than ever. In order for production processes to operate efficiently and smoothly, the devices on the production line must work in harmony with each other. This is where the SMEMA Protocol, widely used in the automation industry, comes into play. In addition to the SMEMA protocol, by using IoT (Internet of Things) technology, devices on the production line can communicate with each other via the internet, making production processes smarter and more efficient. In our blog post, we will talk about what the SMEMA protocol is, its benefits and how it can be used in real-world applications.

What is SMEMA?
SMEMA was created in the 1980s as a non-profit organization to advance standards in the electronics assembly industry and bring equipment manufacturers together. SMEMA (Surface Mount Equipment Manufacturers Association) means the association of surface mount equipment manufacturers. However, today this term has revolutionized the world of industrial automation and has become a standard protocol that enables communication between automation equipment. This protocol is used to ensure harmonious communication between machines on the assembly line and to enable machines from different manufacturers to work together.

What are the benefits?
The SMEMA Protocol has many benefits, some of which are:

• Compatible Integration: It increases the efficiency of the production line and reduces integration costs by ensuring that automation equipment from different manufacturers work in harmony with each other.
• Flexibility and Scalability: SMEMA increases the flexibility of the production line. This means you can quickly adapt to changes made in the production process and easily scale the production line. For example, it becomes much easier to rearrange your production line or add new devices to increase your production capacity.
• Efficiency: Seamless communication between automation equipment makes production processes more efficient. It allows the production line to experience less downtime and reach higher production capacity.
• Standardization: Since the SMEMA Protocol has become a standard in the industry, you can easily install and maintain equipment and systems.
• Synchronization: By providing synchronization between machines using the protocol, it allows the produced parts to be in the right place at the right time and the assembly process takes place smoothly.
• Error Reporting: The SMEMA protocol uses communication messages to report error conditions. Thus, it becomes easier to identify error conditions in control systems and provides information to intervene.

Usage Areas and the Rise of Smart Factories
The SMEMA Protocol is widely used, especially in the electronics manufacturing industry. Surface mount lines, soldering machines, test equipment and other automation equipment are often designed and manufactured in accordance with SMEMA standards. However, the SMEMA Protocol can also be used in other industries. It will be one of the basic building blocks of smart factories, especially by supporting digital transformation. Smart factories use advanced automation, sensors, data analytics and artificial intelligence technologies to optimize production processes, increase efficiency and ensure flexibility. SMEMA Protocol enables the seamless integration of the mentioned components, supporting the operation of smart factories and providing easy integration.

Real life examples

• Automotive Industry: It can integrate different part production machines and assembly lines using the SMEMA Protocol. For example, a transmission production line, an engine assembly line, and a brake system assembly line can communicate with each other and work in coordination using the SMEMA Protocol. Thus, the production process is optimized and product quality is increased.
• Consumer Electronics Industry: A smartphone manufacturer can integrate different production equipment using the SMEMA Protocol. For example, a PCB (Printed Circuit Board) assembly line, a display assembly line and a battery assembly line can be brought together and work in a coordinated manner thanks to the SMEMA Protocol. This makes the phone production process more efficient and ensures that products are delivered on time.

Future Perspective
The SMEMA Protocol will be used in more industries as Industry 4.0 and IoT (Internet of Things) technologies become more widespread. Some real-life examples mentioned show the importance and impact of using the SMEMA Protocol in industrial applications. As a result, this protocol will be used as a powerful tool to optimize production processes and increase efficiency in different industries, while at the same time it will be one of the basic building blocks of smart factories by supporting the digital transformation of production processes. In the future, it is expected that the SMEMA Protocol will become standard in more industries and production processes will be further optimized.

As İLGE Artificial Intelligence, we were honored to participate in the Biotech4SUS event.

During the event, we had the opportunity to discuss the latest developments and innovative projects in the field of biotechnology using artificial intelligence. By exchanging information with many esteemed experts in the field, we gained significant insights into the future of the industry.

We would like to express our gratitude and state that we are proud to take part in such valuable events.

The food production industry is experiencing innovative transformations with the rapid advancement of technology. The most impressive of these transformations occurs with the integration of artificial intelligence (AI) technologies. AI is redefining industrial food production to optimize and increase the efficiency of traditional production methods. So how is AI used in food production?

Artificial Intelligence in Food Production

AI is used in many different fields. To understand how AI technologies are used in food production, let's take a look at the basic principles of these technologies.

1. Collecting Data from Sensors

AI systems can monitor and control production processes in real time by collecting and analyzing data from sensors in production facilities. In this way, possible problems can be predicted and necessary precautions can be taken. For example; A milk processing plant constantly monitors milk quality using AI-powered sensors and automatically stops production when it detects any quality issues.

2. Data Analysis

By analyzing data from production, AI can determine how production processes can be optimized. In this way, production capacity can be increased, energy and raw material usage can be optimized and production costs can be reduced. For example; A juice production plant analyzes production data and optimizes juice blends using AI algorithms. AI determines the most suitable fruit mixtures based on existing fruit stock and demand data, thus better responding to customer demands and minimizing production costs.

3. Automated Process Optimization

AI can automatically optimize production processes, ensuring more efficient and consistent production. In this way, a significant increase in product quality and energy efficiency can be achieved. For example; A bread factory can optimize its production processes using an AI-powered system. Cameras constantly monitor the baking process of the breads, and AI algorithms optimize the operating times and temperatures of the ovens from these images, or AI determines the ideal baking time by analyzing the browning levels of the breads and adjusts the ovens' settings accordingly. In this way, a consistent quality of bread is obtained every time, while energy consumption is also reduced to the optimum level. Additionally, AI constantly learns from new data and constantly improves production processes so that the factory is constantly operating at its most efficient level.

4. Product Design and Innovation

Using AI algorithms, food engineers can create new product formulas and improve the quality of existing products. AI-supported systems can be developed for processes such as the development of special products for analyzing consumer demands. For example; An ice cream company can develop new flavor combinations using AI algorithms. AI analyzes data about customer preferences and market trends and recommends unique ice cream flavors. For example; can develop a new ice cream flavor that successfully combines fruit flavors with chocolate flavors. In this way, the company can offer more diverse and interesting products to customers.

5. Food Safety and Quality Control

AI can monitor and control hygiene standards in food production facilities. It can also increase the quality control of the products and enable the detection of faulty products. In this way, risks to food safety and consumer health can be reduced. For example; Using an AI-powered image recognition system, a frozen food manufacturer can detect damage to product packaging and automatically separate faulty products.

6. Supply Chain Management

AI can be used to optimize the supply chain and reduce food waste. Movements of products in the supply chain can be tracked and stock optimization can be done. For example; Using an AI-powered supply chain management system, a vegetable and fruit processing plant can track its inventory of fresh produce and minimize waste by determining optimal storage conditions.

Challenges and Future Prospects of Artificial Intelligence Technologies

The integration of AI technologies into the food production industry presents some challenges. What does the industry need to do to meet challenges such as big data processing capacity, infrastructure requirements and reliability issues? These challenges and solution suggestions are:

1. Big Data Processing Capacity: AI systems need high processing capacity to process large amounts of data. To solve this problem, production facilities should install high-performance computer systems.
2. Infrastructure Requirements: In order to use AI systems, the necessary infrastructure and expertise must be created. For this reason, universities and private educational institutions should open specialized training programs and courses in the fields of AI and food production, and companies specialized in AI can provide consultancy and solutions to food production facilities.
3. Reliability: It is extremely important that AI systems provide reliable and accurate results. Therefore, AI systems should be tested with real production data and accuracy rates should be determined. In addition, necessary precautions must be taken on issues such as using AI systems in an ethical and transparent manner, data security and algorithm transparency.

Conclusion

The integration of AI technologies in the food production industry is not only a turning point but also the beginning of an exciting journey into the future. These technologies enable revolutionary changes in areas such as optimizing industrial processes, supporting product design and innovation processes, and increasing food safety. However, the difficulties that will be encountered in this transformation process should not be ignored. Issues such as big data processing capacity, infrastructure requirements, and reliability are important considerations that must be addressed to support the successful integration of AI technologies. With the further development of AI technologies in the future, larger and more transformative changes are expected to occur in the food production industry. The future of the food production industry will be shaped brighter and more innovatively with AI.

Performance measurement is essential to machine learning processes because it enables us to assess the efficacy of models created in machine learning on actual data. By evaluating a model's correctness, generalizability, and data fit, performance measurement enables us to assess how successful algorithms and configurations are. Furthermore, performance measurement improves decision-making processes' dependability by assessing how effectively models function in actual scenarios. Performance measurement, then, is essential to the success of machine learning initiatives because it offers an unbiased evaluation of how well-developed models fit with real-world issues. Therefore, in artificial intelligence initiatives, careful performance monitoring and the selection of relevant indicators are essential.

Every machine learning process involves the use of metrics, which are a basic component for assessing the dependability and efficacy of models. Determining the appropriate performance measures is essential for optimizing algorithms and gauging a project's effectiveness. Leading performance measures for both regression and classification issues are examined in this article, along with the insights they offer into model performance. As a result, you can select the metrics that are most relevant for the given use case.

Some of the leading performance metrics include: Accuracy, Precision, Recall/Sensitivity, F1-Score, ROC Curve and AUC (Receiver Operating Characteristic Curve and Area Under the Curve), RMSE (Root Mean Square Error), MAE (Mean Absolute Error), R-Squared.

Some commonly used metrics in machine learning:

1. Accuracy represents the ratio of correct predictions to the total predictions made by a classification model. The accuracy value helps evaluate how well the model is performing from a general perspective. However, accuracy alone may not fully depict the performance of the model because it can be misleading in cases of class imbalance. In other words, it is important to pay attention to the numbers of false positives (FP) and false negatives (FN) alongside true positives (TP) and true negatives (TN) when the model makes classifications.

   True Positive (TP) signifies the instances where the model correctly predicts positive cases that are actually positive. True Negative (TN) indicates the instances where the model correctly predicts negative cases that are actually negative. False Positive (FP) represents the instances where the model incorrectly predicts negative cases as positive. False Negative (FN) indicates the instances where the model incorrectly predicts positive cases as negative.

   The accuracy metric evaluates the model's ability to make correct predictions by considering these four scenarios. However, accuracy alone may be insufficient in cases of imbalanced datasets or cost-sensitive problems. Therefore, it is advisable to use it in conjunction with other metrics to more comprehensively assess the model's performance.

2. Precision measures the proportion of positive instances among the instances that a classification model predicts as positive. Precision focuses on reducing the number of false positive predictions, thus evaluating the model's ability to correctly identify true positives. Precision is particularly important in cases where the cost of false positive predictions is high, such as in medical diagnoses or fraud detection. Therefore, the precision metric should be used to assess the reliability of the model and minimize the number of false positives.
   
   
3. Recall, also known as sensitivity, measures the proportion of true positive instances that a classification model correctly identifies. Recall focuses on reducing the number of false negative predictions and evaluates the model's ability to not miss true positives. Particularly in situations where false negative predictions have serious consequences, such as in medical diagnoses or security applications, recall is crucial. Therefore, the recall metric should be used to assess the model's sensitivity and mitigate the risk of missing true positives.
   
   
4. The F1-Score provides a combined measure of precision and recall performance in a classification model. It balances the effects of both false positives and false negatives, thereby assessing the overall performance of the model more effectively. The F1-Score considers both the accuracy of the model and the risk of missing true positives. Especially in cases of imbalanced classification problems or situations with different costs, the F1-Score should be used.
   
   

5. The Receiver Operating Characteristic (ROC) Curve and Area Under the Curve (AUC) are widely used visual and quantitative metrics for evaluating the performance of a classification model. The ROC Curve is a graph that shows the relationship between sensitivity (recall) and the false positive rate at different thresholds of the classification model. The ROC curve visually represents the performance of the model at various levels of sensitivity and specificity. The selection of thresholds can be used to adjust the model's sensitivity or specificity, providing flexibility in the decision-making process.

   The AUC (Area Under the Curve) represents the area under the ROC curve. AUC condenses the performance of the classification model across all levels of sensitivity and specificity into a single number. The AUC value typically ranges from 0 to 1; a value approaching 1 indicates that the model has excellent performance, while a value approaching 0.5 suggests performance equivalent to random guessing. Therefore, the AUC value is a measure used to assess the overall performance of a classification model.

   ROC Curve and AUC are particularly useful in cases of imbalanced classification problems and situations where different thresholds have varying effects on performance. These metrics provide an important means to understand and optimize the performance of the model across different levels of sensitivity and specificity.

6. RMSE (Root Mean Square Error) is a metric used to evaluate the prediction performance of regression models. RMSE measures the differences between the actual values and the predicted values of the model and calculates the standard deviation of these differences. Particularly in cases where prediction errors are significant, such as in financial forecasting or modeling natural phenomena, RMSE is preferred for evaluation.
   
   
7. MAE (Mean Absolute Error) is a metric used to evaluate the prediction performance of regression models. MAE calculates the average of the absolute differences between the actual values and the predicted values of the model. Particularly in cases where there are outliers, MAE may be preferred over RMSE because it is more resistant to outliers.
   
   
8. R-Squared (R²) is a metric used in regression models that expresses the proportion of the variance in the dependent variable that is explained by the independent variables. R-Squared indicates how well a model fits the data; a high R-Squared value indicates that the model fits the data well, while a low R-Squared value suggests that the model's ability to fit the data is weak. Therefore, R-Squared is used to evaluate and compare the performance of regression models.

To sum up, performance monitoring is essential to assessing if machine learning initiatives are successful. We can assess the built models' effectiveness on real-world data by using well chosen indicators. Selecting appropriate performance indicators enables us to comprehend the models' precision, capacity for generalization, and data fit. Accuracy, precision, recall, F1-Score, ROC Curve and AUC, RMSE, MAE, and R-Squared are some of the top performance measures. These metrics offer distinct insights and are crucial for situations involving both regression and classification. Making the right measurement choices is essential to gauging the project's success and enhancing your algorithms. Thus, in machine learning initiatives, careful performance measurement and the selection of relevant indicators are required.

While machine learning forms a strong foundation of data-driven systems, obtaining labeled data is often time-consuming and costly. This limitation has led to the emergence of techniques such as semi-supervised learning. These techniques aim to improve model performance by leveraging large amounts of unlabeled data as well as limited amounts of labeled data. In this article, we will examine what semi-supervised learning is, how it works, and how it can be used in real-world applications.

What is Semi-Supervised Learning?

Semi-supervised learning is a machine learning approach that bridges unsupervised learning and supervised learning, where models are trained using both labeled and unlabeled data. This method significantly increases the performance of models by taking advantage of large amounts of unlabeled data in cases where labeled data is limited.

What are the advantages?

• Reduces the cost of labeling: Instead of labeling large data sets, it is sufficient to label only a small portion of them. In this way, significant savings are achieved in time and labor costs.
• Uses more data: By using unlabeled data, it allows the model to be trained with a wider range of information.
• Can solve more complex problems: By solving patterns in unlabeled data, it helps the model discover previously unnoticed patterns.
• Generalization Ability: Semi-supervised learning allows the model to learn from more general and diverse data distributions. This allows the model to generalize better and perform better on real-world data.

How does it work?

Semi-supervised learning usually occurs in two steps: First, an initial model is trained with some labeled data, and then this model improves itself using unlabeled data. Various techniques can be used in this process, such as pseudo-labeling, self-training, and graph-based methods.

• Pseudo-Labeling: It is a technique in which instead of manually labeling unlabeled data, the model is given approximate labels based on labeled data. The process steps are as follows:

In the first step, the model is trained with labeled data.

Then, pseudo-labels are given to the unlabeled data using the trained model. At this stage, the resulting predictions are called "pseudo-labels" because they are derived from the original labeled data.

Finally, the model is trained again with these pseudo-labels.

This process is repeated until the model's performance increases and it reaches higher levels of accuracy.

• Self-Education: It is similar to pseudo-labeling, but with a difference: In self-training, the model only has a high confidence level (For example, when a model predicts an image of a bird with a 60% probability, the confidence level of that prediction is expressed as 60%, meaning that the model thinks that prediction is correct with a 60% probability). (shows.) owner's predictions are accepted. Additionally, by repeating this process several times, the performance of the model is further improved. So how does it work?

First, a limited number of labeled data samples are selected. This small data set is trained with      traditional supervised learning methods to create an initial model.

Then, using the partially trained model, predictions are generated for the remainder of the yet unlabeled dataset. Again, the predictions obtained here are called "pseudo-labels".

If any pseudo-labels exceed this confidence threshold, these labels are added to the labeled dataset and used to train an improved model with a combined dataset.

This process occurs by adding increasing amounts of pseudo-labels over several iterations. When the data is suitable, the model's performance will continue to improve with each iteration.

• Graph Based Methods: In these methods, data is represented in a graph structure and points with similar properties are connected to each other. Starting from labeled samples, labels are assigned to unlabeled data using the graph structure. The model is retrained using the newly labeled data. This process is repeated until the performance of the model improves.

Real World Applications

Semi-supervised learning can be widely used in many real-world application areas, thanks to the abundance and ease of access of unlabeled data. For example, if we have a large amount of unlabeled data and a limited number of labeled data, it may make sense to use various semi-supervised algorithms to solve the problem. For example, it can be used in various application areas such as medicine (disease detection), image processing (object recognition, face recognition), natural language processing (language translation, text classification).

Potential Applications in Business Scenarios

• Analysis of customer emotions in e-commerce

Let's say an e-commerce company wants to analyze reviews to understand customers' feelings and preferences about products. Most existing comments are untagged. Those that are labeled constitute only a small portion. In this case, semi-supervised learning can be used. First, a sentiment analysis model is trained with a small set of labeled data. This model is then used to make predictions on untagged comments. The predictions are then validated manually, and the validated ones are added to the labeled dataset. With this verified data, the model is retrained and its performance is improved. This process is repeated to refine the model with more labeled data and analyze a wider range of comments. As a result, the company can evaluate customer feedback more effectively and make better decisions to improve its products..

• Driverless vehicle development

Driverless vehicles perceive their environment through various sensors and drive safely by processing this data. However, it is very costly to label and train the model for every situation that driverless vehicles may encounter. In this case, semi-supervised learning comes into play. The company builds a driverless vehicle model starting from a limited number of labeled data. This model is used to predict driving scenarios using unlabeled data derived from real-world data. Experts correct the model's erroneous predictions and add the validated scenarios to the labeled dataset. With this verified data, the model is retrained and its performance is improved. This process allows the model to be enhanced with more labeled data. In this scenario, semi-supervised learning helps autonomous vehicles drive safer while also reducing training costs.

Conclusion

Semi-supervised learning has great potential to expand the boundaries of artificial intelligence. This method is used as a powerful tool to improve the performance of machine learning models when labeled data is limited. However, semi-supervised learning is not perfect like other algorithms. The results provided by the algorithm are less reliable and not 100% reliable because there is no way to verify whether it produces 100% correct labels. As a result, each algorithm has its own advantages and disadvantages and it is important to choose the appropriate algorithm for the problem.

In the ever-evolving landscape of artificial intelligence, Reinforcement Learning (RL) stands as a powerful force, propelling breakthroughs across diverse domains such as autonomous systems, robotics, game playing, and beyond. Rooted in machine learning, RL represents a dynamic approach wherein an intelligent agent learns to make decisions by actively engaging with an environment, receiving feedback in the form of rewards or penalties. This interactive process enables the agent to iteratively refine its decision-making prowess, paving the way for unparalleled advancements in the realm of smart, adaptive systems. This characteristic behavior echoes the fundamental aspects of human nature. Mankind is willing to get rewards and avoid any penalties. This dilemma of decisions is the one who provides feedbacks.

Reinforcement Learning is characterized by a unique set of core components, each playing a crucial role in the learning process. At its core, RL involves an agent, the decision-maker, interacting with an environment, a contextual space where decisions are required. The agent navigates this environment by perceiving its current state, selecting actions based on predefined strategies, executing these actions, and receiving feedback in the form of rewards or penalties. The ultimate objective is to iteratively enhance the agent's decision-making capabilities over time, optimizing for the accumulation of rewards.

This article embarks on a journey to unravel the intricacies of Reinforcement Learning, exploring its foundational elements, the intricacies of its learning process, and the pivotal role it plays in cutting-edge applications. We delve into the challenges that permeate the RL landscape and illuminate recent advancements that continue to push the boundaries of what is achievable. Join us in this comprehensive exploration of Reinforcement Learning, as we dissect its components, analyze its applications, confront its challenges, and chart the course for its promising future in the field of artificial intelligence.

Core Components of Reinforcement Learning:

Agent: At the heart of RL is the agent, an intelligent entity tasked with making decisions within a given environment.
Environment: The environment represents the external system with which the agent interacts. It could be a physical space, a virtual simulation, or any context where decisions need to be made.
State: A state is a snapshot of the environment at a particular time, providing crucial information for the agent to make decisions.
Action: The set of possible moves or decisions the agent can take in a given state.
Reward: A numerical feedback signal from the environment, indicating the immediate outcome of the agent's action. The objective is to maximize cumulative rewards over time.
Policy: The strategy or set of rules the agent follows to determine its actions based on the current state.

The RL process is characterized by the agent interacting with the environment in a series of discrete time steps. The general flow involves the following:

Observation: The agent observes the current state of the environment.
Decision-Making: The agent selects an action based on its current policy.
Action Execution: The chosen action is executed in the environment.
Reward Reception: The agent receives a reward or penalty based on the outcome of its action.
Learning: The agent updates its policy based on the received feedback, aiming to improve its future decision-making.

Applications of Reinforcement Learning:

Game Playing: RL has demonstrated remarkable success in mastering complex games, such as Go, Chess, and video games, surpassing human performance.
Robotics: Autonomous robots use RL to learn and adapt to their surroundings, making them more versatile in real-world scenarios.
Finance: RL is employed in algorithmic trading and portfolio optimization, where agents learn to make profitable investment decisions.
Healthcare: RL assists in personalized treatment plans, drug discovery, and optimizing resource allocation in healthcare settings.
Autonomous Vehicles: RL plays a crucial role in training self-driving cars to navigate complex traffic scenarios and make optimal decisions.

Challenges in Reinforcement Learning:

Exploration-Exploitation Dilemma: Balancing the need to explore new actions and exploit known ones is a fundamental challenge in RL.
Credit Assignment Problem: Determining which actions contributed to a particular outcome is challenging when there is a delay between the action and the received reward.
Complexity and Scalability: As RL models become more sophisticated, training them on large-scale environments poses computational challenges.

Recent Advancements:

Deep Reinforcement Learning (DRL): Integration of deep neural networks with RL, enabling agents to learn complex representations and achieve human-level performance in various tasks.
Proximal Policy Optimization (PPO): An algorithm that optimizes policy functions in a stable and efficient manner, addressing some of the challenges in traditional policy gradient methods.
Multi-Agent Reinforcement Learning: Extending RL to scenarios with multiple interacting agents, fostering collaboration and competition.

Conclusion:

Reinforcement Learning stands as a powerful paradigm for training intelligent agents to make decisions in diverse and dynamic environments. As advancements in algorithms and computational resources continue, the applications of RL are poised to revolutionize industries and contribute to the development of more capable, adaptable, and autonomous systems.

Today, artificial intelligence is a powerful tool that is being used in many different industries. It has grown quickly. But in many applications, problems like limited data, inadequate processing capacity, or time limits frequently arise. This is the context in which transfer learning is useful. By using pre-trained models' expertise for new tasks, transfer learning helps overcome challenges brought on by a lack of data and processing resources. We shall discuss transfer learning's definition, methodology and significance in artificial intelligence projects.

When our resources for data or computing power are limited, transfer learning is especially helpful. By utilizing the expertise of previously trained models, it offers more suitable beginning points for new tasks and frequently produces satisfactory outcomes with minimal amounts of labeled data.

When it comes to discussing the application steps of transfer learning, the selection of a pre-trained model is the first and fundamental step. Typically, a model trained with large datasets for extended periods and associated with a similar task is chosen. The second step in transfer learning is determining the layers to be transferred. Early and middle layers of the pre-trained model are usually preferred in this determination process as these layers have been trained to recognize general features and patterns. The third step involves retraining the selected model's final layers according to the target task's dataset. This allows the model to learn specific information required for the new task. The final step is evaluating the model's performance on the new task. In this step, fine-tuning can be performed to improve the model's success, or the training process can be repeated. If we delve into these steps in more detail:

• Model Selection: An architecture that bears a strong connection to the intended job is selected. For picture classification tasks, for example, architectures such as ResNet, MobileNet, or VGG16/19 can be applied. During training, time is saved by downloading pre-trained weights for certain architectures. To further explain, a model can be developed for a particular task, like truck defect detection and this model can be quickly retrained on truck images by using the feature extraction layers of a model trained on the ImageNet dataset, which contains millions of data and has been trained for numerous tasks.                                                           
• Freezing Layers: In the freezing layers stage, the pre-trained model's early layers or its feature extraction layers are usually frozen. When layers are frozen, no learning takes place throughout the training process since their weights are not updated and these layers stay fixed. For instance, the early layers of an image classification model are frequently employed to teach the model fundamental properties like corners and edges. Most of the time, these characteristics can be applied to many tasks. Therefore, fundamental characteristics are maintained by freezing these layers.  
• Adding New Layers: These layers are frequently added to the model's initial architecture later on, usually in the final layers (output layers) and are used to anticipate or learn the model's specialized duties. Based on the model's previously learned characteristics, these new layers seek to learn tailored features to more effectively complete a specific task. They usually consist of the output layers or classification layers of the model since they are usually utilized to forecast a task's result. For instance, let's say we have a model trained to identify a wide range of animal species and we wish to use it to classify only cats and dogs. In this instance, our goal may be achieved by simply lowering the classifier layer's number of classes to two. 
• Training New Layers: After the output layers are modified, the model is retrained. With the new output layers, the model learns the new task. During the retraining process, the feature extraction layers of the pre-trained model are typically frozen and only the newly added output layers are trained. Therefore, while preserving pre-learned general features, the model's customized classification abilities for the new task are developed. 
• Fine-tuning: This is the final stage in the transfer learning process and is typically used to further improve the model's performance. This stage involves making the pre-trained model more suitable for the target task. To briefly explain the steps of fine-tuning, it includes operations aimed at enhancing the model's performance such as adjusting the weights of the pre-trained model, using lower learning rates, data augmentation and employing regularization techniques.

The basic stages of transfer learning have been briefly covered here: choosing a pre-trained model, identifying the layers to be transferred (freezing layers), adding new layers, training the newly added layers and lastly assessing the model's output and adjusting as needed. The aforementioned procedures are essential to the effective application of transfer learning.

Transfer learning has many applications across various industries. For instance, successful results have been obtained using transfer learning in areas such as image recognition, natural language processing (NLP), medical imaging and the automotive industry. In the future, it is anticipated that transfer learning will not only reduce training costs in the field of artificial intelligence but also expand its scope for more widespread use.

In conclusion, transfer learning emerges as an important technique in the field of artificial intelligence. When faced with situations where data is limited, computational power is restricted, or time is constrained, leveraging the knowledge of pre-trained models to provide a starting point for new tasks is highly valuable. When applied with the right strategies, maximum benefit can be derived from the accumulated knowledge of pre-trained models, leading to faster and more efficient solutions for new tasks.

Neural networks, unlike their traditional structures, play a significant role in machine learning, where the output from the previous step is used as input for the current step, as in the case of Recurrent Neural Networks (RNNs). In traditional neural networks, all inputs and outputs are independent of each other. However, in situations like predicting the next word in a sentence, there is a need for information from previous words, necessitating the ability to remember them. RNNs were developed to meet this need and solve this problem using a hidden layer.

The fundamental and most important feature of RNN is its hidden state. This hidden state is a feature that remembers information about a sequence and is also called the Memory State because it recalls the previous network input. RNN uses the same parameters for each input and performs the same task for all inputs or hidden layers, reducing parameter complexity compared to other neural networks.

However, one of the problems RNN faces is its difficulty in handling long-term dependencies effectively. To address this issue, Long Short-Term Memory (LSTM) was developed. LSTM enhances the memory state of RNN, effectively dealing with long-term dependencies.

LSTM Gates and the Art of Sequential Data Processing
Neural networks, especially Recurrent Neural Networks (RNNs) and their enhanced version, Long Short-Term Memory (LSTM), have achieved significant success in processing sequential data. In this article, we will take a deep mathematical look at the forgetting, input, and output gates, which are the backbone of LSTM.

Forgetting Gate
The forgetting gate is a process that determines how much of the previous information will be remembered. This process can be expressed with the following formulas:

${�}_{�}=�({�}_{�}\cdot [{ℎ}_{�-1},{�}_{�}]+{�}_{�})$

${�}_{�}={�}_{�}\cdot {�}_{�-1}$

Where ${�}_{�}$ is the output of the forgetting gate, ${�}_{�}$ is the weight matrix, ${ℎ}_{�-1}$ represents the previous memory state, ${�}_{�}$ represents the current input, and ${�}_{�}$ is the bias term.

Input Gate
The input gate is critical for updating the model's current state and adding new information. This process can be expressed with the following formulas:

${�}_{�}=�({�}_{�}\cdot [{ℎ}_{�-1},{�}_{�}]+{�}_{�})$

${\widetilde{�}}_{�}=\tanh ({�}_{�}\cdot [{ℎ}_{�-1},{�}_{�}]+{�}_{�})$

${�}_{�}={�}_{�}\cdot {\widetilde{�}}_{�}$

Where ${�}_{�}$ is the output of the input gate, ${\widetilde{�}}_{�}$ represents the new information, ${�}_{�}$ and ${�}_{�}$ are the weight matrices, ${�}_{�}$ and ${�}_{�}$ are the bias terms.

Output Gate
The output gate is a crucial stage that brings the learned information of LSTM to the outside world. This process can be expressed with the following formulas:

${�}_{�}=�({�}_{�}\cdot [{ℎ}_{�-1},{�}_{�}]+{�}_{�})$

${ℎ}_{�}={�}_{�}\cdot \tanh ({�}_{�})$

Where ${�}_{�}$ is the output of the output gate, ${ℎ}_{�}$ represents the new memory state, ${�}_{�}$ is the weight matrix, and ${�}_{�}$ is the bias term.

These formulas mathematically explain the internal workings of each LSTM gate, enabling the model to effectively process relationships in sequential data.

APPLICATIONS OF RNN Natural Language Processing (NLP)
RNNs are used in tasks such as language modeling, text classification, translation, and speech recognition in natural language processing. Their ability to process sequential data makes them ideal for understanding context within sentences and tracking the evolution of language over time.

Time Series Analysis
RNNs are applied in analyzing time series data for tasks like financial market predictions, weather forecasts, and stock management. RNNs can predict future values by utilizing information from previous time points.

Image Processing
RNNs can be used in visual data processing tasks such as video analysis, object recognition, and video captioning. They are particularly useful for tracking video streams or moving objects.

Speech Processing
RNNs are employed in tasks like speech recognition, speech synthesis, and emotion analysis. Their ability to work with sequential data is advantageous for processing dynamic data like speech.

Genomic Data Analysis
In bioinformatics, RNNs are applied to analyze genetic data. They are useful for recognizing patterns in sequential biological data such as DNA or RNA sequences and understanding gene functions.

Music Composition
RNNs can create new musical compositions by learning relationships between musical notes. They excel at understanding and generating musical patterns over time.

Game Development
RNNs are used to create AI-based game characters and manage interactions within games. They can predict future actions based on players' past behaviors.

Energy Consumption Prediction
RNNs can be utilized in the energy sector for tasks like predicting electricity consumption, energy demand forecasting, and optimizing energy efficiency.

These applications demonstrate how the ability of RNNs to process sequential data and temporal dependencies can be leveraged across various industries.

CONCLUSION
Recurrent Neural Networks (RNNs) offer unique power in modeling dependencies in sequential data. Their ability to associate past with future makes them successful in language analysis, time series forecasting, and many other applications. However, the challenge of RNNs in effectively managing long-term dependencies has led to the emergence of advanced structures like LSTM. In this article, you have gained an understanding of the fundamental structures of RNN and witnessed its evolution over time. However, it's important to remember that each model has its advantages and limitations. The foundational understanding provided by RNN can be a step towards managing complexity and tailoring your model to your specific task. In this exciting world, with the deepening of sequential data, the doors to predicting the future are opening, thanks to structures like RNNs.

REFERENCE
"Recurrent Neural Networks (RNN) - A Comprehensive Guide." GeeksforGeeks. https://www.geeksforgeeks.org/recurrent-neural-networks-rnn/ (Erişim Tarihi 25, 2023).