{"id":2328,"date":"2025-06-11T14:34:00","date_gmt":"2025-06-11T09:04:00","guid":{"rendered":"https:\/\/texpertssolutions.com\/notes\/?p=2328"},"modified":"2025-06-26T14:50:47","modified_gmt":"2025-06-26T09:20:47","slug":"what-is-activation-function","status":"publish","type":"post","link":"https:\/\/texpertssolutions.com\/notes\/2025\/06\/11\/what-is-activation-function\/","title":{"rendered":"What is Activation Function?"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\ud83e\udde0 What is an <strong>Activation Function<\/strong>?<\/h2>\n\n\n\n<p>An <strong>Activation Function<\/strong> is a <strong>tiny math rule<\/strong> used inside a neural network.<br>It tells each &#8220;neuron&#8221; in the network:<br>\ud83d\udc49 <strong>Should I be active or not?<\/strong> \ud83d\udd1b\ud83e\udd16<\/p>\n\n\n\n<p>Think of it like a <strong>light switch<\/strong> \ud83d\udca1 \u2014 the function decides if a signal should pass through (turn on) or not (stay off).<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\udd79\ufe0f Why do we need it?<\/h3>\n\n\n\n<p>Without activation functions, your model would just be a bunch of boring math equations that can\u2019t learn anything interesting \ud83d\ude34<\/p>\n\n\n\n<p>But <strong>with activation functions<\/strong>, your model can:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Learn patterns \ud83c\udfa8<\/li>\n\n\n\n<li>Make decisions \ud83d\udca1<\/li>\n\n\n\n<li>Understand language, images, sound \u2014 everything! \ud83c\udf0d<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\u2699\ufe0f How does it work?<\/h3>\n\n\n\n<p>When a neuron gets a number (input), the activation function:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\ud83e\uddee Takes that number<\/li>\n\n\n\n<li>\ud83d\udccf Applies a rule (like \u201ckeep only positive numbers\u201d)<\/li>\n\n\n\n<li>\ud83d\udce4 Sends the result to the next layer<\/li>\n<\/ol>\n\n\n\n<p><a href=\"https:\/\/qbase.texpertssolutions.com\/index.php\/Example_of_ReLU_Activation_Function\">https:\/\/qbase.texpertssolutions.com\/index.php\/Example_of_ReLU_Activation_Function<\/a><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83e\uddea Most Common Activation Functions (with emojis!)<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Name<\/th><th>Emoji<\/th><th>What It Does<\/th><th>Output Range<\/th><\/tr><\/thead><tbody><tr><td><strong>ReLU<\/strong> (Rectified Linear Unit)<\/td><td>\u26a1\ud83d\udd0b<\/td><td>Turns <strong>negatives into 0<\/strong>, keeps positives as they are<\/td><td>0 \u27a1\ufe0f \u221e<\/td><\/tr><tr><td><strong>Sigmoid<\/strong><\/td><td>\ud83d\udfe2\ud83d\udcc9<\/td><td>Squeezes numbers into a range between <strong>0 and 1<\/strong><\/td><td>0 \u27a1\ufe0f 1<\/td><\/tr><tr><td><strong>Tanh<\/strong><\/td><td>\ud83c\udf08\ud83d\udd04<\/td><td>Like Sigmoid, but gives values between <strong>-1 and 1<\/strong><\/td><td>-1 \u27a1\ufe0f 1<\/td><\/tr><tr><td><strong>Softmax<\/strong><\/td><td>\ud83e\udde0\ud83d\udcca<\/td><td>Turns numbers into <strong>probabilities<\/strong> (used for classification)<\/td><td>0 \u27a1\ufe0f 1 (all add up to 1)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83e\udde9 Real-Life Analogy:<\/h3>\n\n\n\n<p>Imagine you&#8217;re tasting a spicy dish \ud83c\udf36\ufe0f<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>If it&#8217;s <strong>mild<\/strong>, you say \u201cokay\u201d (low signal)<\/li>\n\n\n\n<li>If it\u2019s <strong>very spicy<\/strong>, your brain says \u201c\ud83d\udd25 HOT!!\u201d (high signal)<\/li>\n\n\n\n<li>If it&#8217;s <strong>not spicy<\/strong>, your brain says \u201cmeh\u201d (zero signal)<\/li>\n<\/ul>\n\n\n\n<p>That\u2019s what an <strong>activation function<\/strong> does \u2014 it <strong>decides how strongly to react<\/strong> to an input.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\udcd8 Simple Summary:<\/h3>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><strong>Activation Function = the rule that tells the neuron how to respond<\/strong> \ud83e\udde0\ud83d\udece\ufe0f<br>It helps neural networks make smart decisions and understand the world better.<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\ud83e\udde0 What is an Activation Function? An Activation Function is a tiny math rule used inside &hellip; <a title=\"What is Activation Function?\" class=\"hm-read-more\" href=\"https:\/\/texpertssolutions.com\/notes\/2025\/06\/11\/what-is-activation-function\/\"><span class=\"screen-reader-text\">What is Activation Function?<\/span>Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":2345,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[641],"tags":[],"class_list":["post-2328","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-machine-learning"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/texpertssolutions.com\/notes\/wp-content\/uploads\/2025\/06\/2.png?fit=1280%2C720&ssl=1","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/posts\/2328","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/comments?post=2328"}],"version-history":[{"count":3,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/posts\/2328\/revisions"}],"predecessor-version":[{"id":2361,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/posts\/2328\/revisions\/2361"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/media\/2345"}],"wp:attachment":[{"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/media?parent=2328"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/categories?post=2328"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/texpertssolutions.com\/notes\/wp-json\/wp\/v2\/tags?post=2328"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}